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INTRODUCTION 


The purpose of this report is to describe findings from analyses conducted on data 
collected from the IDEA Feedback System for Chairs (FSC) from 2013 to 2016. The report begins 
with an explanation of the purpose of the system and the current structure of the Chair Self- 
Assessment (CSA) and Faculty Perceptions of Chair (FPC) instruments. This is followed by a 
description of the 2013 to 2016 database, results of statistical analyses, and empirical evidence that 
supports the reliability and validity of the instruments. 


A unique feature of the IDEA FSC is a gap analysis that compares the academic 
department chair's self-ratings with faculty ratings of the chair's performance of relevant 
administrative responsibilities, personal characteristics, and administrative methods. The system 
is based on a model of reflective practice (see Figure 1), which is consistent with the view that 
chairs are more likely to improve if they receive feedback that motivates them. Although mere 
use of the feedback system does not guarantee improvement, meaningful change begins by 
collecting feedback tailored specifically to prioritized responsibilities. With the aid of 
recommendations made in the IDEA Chair Report, chairs interpret the results and identify 
strengths and areas for improvement. They can then learn more about how to become a better 
department chair by reading any of the hundreds of books and articles published on the topic or 
by participating in professional development. The chair is also encouraged to reflect on what is 
learned and discuss the report with a trusted colleague. The desired outcome of that discussion 
should be a specific plan for improvement. 


Figure 1 
Model of Reflective Practice 


Collect 
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Pe Read & 4 
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As stated in IDEA Technical Report No. 14 (Benton, Gross, Pallett Song, & Webster, 
2010, p. 2), 


The IDEA Center discourages high-stakes use of its instruments. As with any effective 


evaluation system, multiple sources of information should be collected, analyzed, and 
interpreted. Making a summative decision about a chair’s employment status based 
solely on the results of the IDEA [FSC] would run counter to its intent. The IDEA 
system is, therefore, not intended to be used as the sole source of input for either 
formative or summative evaluation. However, its unique focus on feedback about the 
performance of relevant responsibilities and associated personal characteristics and 
administrative methods makes it an extremely useful tool for developmental purposes. 


Background 


The IDEA FSC descends from the IDEA Feedback for Department Chairs System 
(FDCS), which was preceded by the Departmental Evaluation of Chairperson Activities for 
Development (DECAD) program developed at Kansas State University in the 1970s 
(McCarthy, 1972; Hoyt 1976). See Benton et al. (2010) for a description of how the DECAD, 
FDCS, and the FSC were developed. 


The Current Feedback System for Chairs 


Construction of the current instruments in the FSC was guided, in part, by statistical 
analyses performed on the 2003 to 2007 database (see Benton et al. (2010) for a description). 
The FSC features two instruments and a report. On the CSA, chairs use the scale of 1 = Low 
Priority, 2 = Moderate Priority, and 3 = High Priority to indicate the level of priority within 
their unit for each of 21 responsibilities. They can then choose to rate themselves on their 
performance of those responsibilities over the past year, using the scale of 1 = Poor, 2 = Fair, 3 
= In Between, 4 = Good, and 5 = Outstanding. They may also rate themselves on 11 personal 
characteristics and 21 administrative methods by responding with 1 = Definite Weakness, 2 = 
More a Weakness Than a Strength, 3 = In Between, 4 = More a Strength Than a Weakness, or 5 
= Definite Strength. 


A key element of the FSC is an optional gap analysis made possible by faculty and chairs 
responding to the same survey items. The FPC instrument contains the 21 administrative 
responsibilities, 11 personal characteristics, and 21 administrative responsibilities along with the 
scales the chairs use for self-ratings. Faculty are given the additional option of Omit Response. 
Lastly, faculty members respond to two questions on the FPC that assess their overall summary 
judgments of the chair, using the scale 1 = Strongly Disagree to 5 = Strongly Agree. 


On the Chair Report, the average (mean) faculty rating for each item is then subtracted 
from the respective chair's self-rating. The difference between ratings represents the "gap" 
between self- and faculty-perceptions and is used to focus improvement efforts on 
administrative responsibilities, personal characteristics, and administrative methods where a 
positive gap exists (i.e., when the self-rating is higher than the average faculty rating). 


The Chair Report also presents average faculty ratings and percentage of positive 


ratings on responsibilities the chair assigns as a moderate or high priority’. Faculty ratings are 
also reported for personal characteristics and administrative methods highly correlated with 
successful performance of each relevant responsibility. For summative purposes, mean faculty 
ratings are given for two summary judgment items. Finally, faculty written comments about 
the chair and department are included. 


DESCRIPTION OF THE 2013 TO 2016 DATABASE 


Although the FSC instruments were revised in 2010, the updated system was not 
implemented until 2013. From 2013 to 2016, 16,443 faculty members rated a total of 1,219 
chairs at 54 different institutions. Several exclusion criteria were used to create the final sample 
used for the analyses found in this report. First, we excluded chair ratings based on the number 
of faculty raters. Previous analyses revealed that reliability is unacceptably low if there are fewer 
than eight raters (Hoyt et al., 1999). However, the current database showed acceptable levels of 
reliability for chair ratings with five or more (see Appendix A). Thus, we excluded chair ratings 
with fewer than five raters (n = 116). Second, if the department chair had multiple ratings from 
2013 to 2016, only one rating was retained by random selection to ensure independence among 
observations (370 cases excluded). Third, a faculty member’s data were excluded if he/she 
responded to no more than 50% of the items on the FPC (n = 54). The final sample consisted of 
11,169 faculty members rating 762 different chairs from 54 institutions. 


Descriptive Information about the Departments and Institutions 


Tables 1, 2, and 3 provide characteristics of the academic departments and their 
institutions. As shown in Table 1, the number of faculty who were sent surveys ranged from 5 to 
356 with an average of about 20. The mean percentage faculty response rate was 79% (0.79 x 
100). Table 2 shows the percentage of institutions representing various regional accreditors. The 
largest percentage represented the North Central Association of Colleges & Schools region, 
whereas the institutions from Northwest Commission on Colleges & Universities were least 
represented. As shown in Table 3, the majority of institutions (69%) were public. 


Table 1 

Descriptive Statistics for Department Size and Faculty Response Rate (N = 762) 
Variable Minimum Maximum Median IQR M SD 
Faculty surveyed 5 356 16 12. 19.62 18.87 
Response rate 0.22 1.00 0.79 0.14 


Note. IQR = Interquartile range 


| See https://prod.iol3.org/r/sample/chair/1 for an example Chair Report. 
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Table 2 

Number and Percent of Institutions Representing Various Regional Accreditors 
Region n % 
Middle States Assoc. of Colleges & Schools 9 16.6 
North Central Assoc. of Colleges & Schools 21 38.8 
Northwest Commission on Colleges & Universities 2 Bee 
Southern Assoc. of Colleges & Schools 17 31.5 
Western Assoc. of Schools & Colleges ) 9.3 

Note. N = 54. 

Table 3 

Number and Percent of Public and Private Institutions 
Institution type n % 
Public 3768.5 
Private 17 315 

Note. N = 54. 


Descriptive statistics about student enrollment, annual tuition, number of fulltime staff, 
and proportion of fulltime faculty are shown in Table 4. On average, institutions in our sample 
had a student enrollment of 15,206, an annual tuition of $15,634, 3,476 fulltime staff, and 71% 
of faculty in fulltime appointments. 


Table 4 

Descriptive Statistics of Institutional Characteristics (N = 54) 
Variable Minimum | Maximum Median M SD 
Student enrollment 705 58,322 11,928 15,206 12,587 
Annual Tuition (in dollars) 3,543 45,300 9,714 15,634 11,999 
Fulltime staff 134 27,810 1,865 3,476 4,675 
Proportion of fulltime faculty 0 1.00 0.71 0.71 0.21 


Table 5 shows the number and frequency of institutions by Carnegie classification. Most 
institutions were doctoral-granting institutions; associates-degree institutions were least 
represented. 


Table 5 

Number and Percent of Institutions by Carnegie Classification 
Degree n % 
Associate 2 rd 
Baccalaureate 5 9.3 
Masters 1 35.2 
Doctoral 28 ei eis) 


Note. N = 54. 
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Characteristics of the Chairs and Faculty 


Table 6 presents statistics on the Carnegie classification of chairs’ institutions. Most 
chairs were in an institution that awarded graduate degrees, with approximately 62% awarding 
the doctorate and 31% master’s degrees. Fewer chairs (6.5%) came from institutions offering 
only undergraduate degrees (i.e., associate or baccalaureate degrees). 


Table 6 

Number and Percent of Chairs by Institution Carnegie Classification 
Degree n % 
Associate 10 1.3 
Baccalaureate 40 52 
Doctoral 470 61.7 
Masters 233 30.6 
No Response 9 1.2 

Note. N = 762. 


As shown in Table 7, two thirds of chairs (66%) had been in their position for no more 
than five years. Table 8 shows that about half of chairs (52%) were appointed by an internal 
institutional search either within or external to the department. Most chairs (93%) reported that 
50% or fewer of the courses within their department were taught by adjunct faculty (see Table 
9). As shown in Table 10, slightly more than half of chairs (57%) did not know if adjunct faculty 
were included in the survey process. 


Table 7 

Number and Percent of Chairs Reporting Various Years of Service, Counting the Current Year 
Number of years n % 
One or less 115 15.1 
Two to three 228 30.0 
Four or five 158 20.7 
Greater than five 258 33.8 
No response 3 0.4 

Note. N = 762. 

Table 8 

Number and Percent of Chairs Hired by Type of Search 
Appointment method n % 
Internal search within the department 339 44.5 
Internal search within the institution but external to the department 57 7.5 
External search to the institution 283 37.1 
Other 82 10.8 
No Response 1 0.1 


Note. N = 762. 
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Table 9 

Ranges of Percent of Courses in Department Taught by Adjunct/Part-time/Contingent Faculty 
Percent of courses n % 
None 65 8.5 
1- 5% 100 13.1 
6-10% 135 L757 
11-25% 196 25.6 
26-50% 199 26.0 
>51 52 6.8 
No Response 15 0.4 

Note. N = 762. 

Table 10 


Number and Percent of Adjunct/part-time/contingent Faculty Members Included in The Survey 
Process 


Participation description n % 

All adjunct faculty were invited to participate 103 13.5 
Only some adjunct faculty were invited to participate 52 6.8 
No adjunct faculty were invited to participate 128 16.8 
I don’t know 435 Seek 
Not applicable, we don’t have adjuncts in my department 40 52 
No Response 4 0.5 

Note. N = 762. 


Group Comparisons in Chair Ratings of Administrative Priorities 


We used the Crosstabs procedure in SPSS to make group comparisons of chair priority 
ratings on each of the 21 administrative responsibilities by the following grouping variables: 
Carnegie classification, institutional control (public vs. private institution), and chair years of 
service. With a Bonferroni adjustment, each chi-square was conducted at a = .05/21 = .0024. 
When expected cell counts were less than 5, we conducted Fisher’s exact test, which finds the 
exact probability value statistics to confirm chi-square statistics. Following significant chi-square 
tests, z-tests of column proportions (a = .05) were conducted to identify differences in column 
proportions for each chair priority rating (i.e., low priority, moderate priority, and high priority). 


Table 11 shows frequencies and percentages of chair priority ratings by Carnegie 
classification. Although descriptive statistics for associate degree institutions appear in the table, 
we did not include this group in the Crosstab analysis due to its small sample size (n = 10). Cells 
within rows that had different subscripts differed significantly in percentage values. A higher 
percentage of chairs from doctoral-granting institutions assigned a high priority to securing 
external funding, recruiting promising faculty, stimulating research, fostering development of 
faculty talents, and rewarding faculty than chairs in masters-and baccalaureate-granting 
departments. In contrast, a higher percentage of chairs from masters than bachelors and doctoral 
institutions made acquainting new faculty a high priority. Finally, chairs from baccalaureate and 
masters-granting institutions placed a higher priority on attending to administrative tasks than 
chairs from doctoral-granting institutions. 
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We also found several differences in chair priority ratings by institutional control (see 
Table 12). A higher percentage of chairs from public institutions assigned a high priority to 
obtaining grants, recruiting promising faculty, stimulating research, and rewarding faculty than 
did chairs from private institutions. In contrast, a higher percentage of chairs from private 
institutions emphasized developing collegiality, acquainting new faculty, communicating 
administrative expectations, facilitating curriculum development, and assessing student learning. 


We found only one difference in chair priority ratings by the number of years served. As 
shown in Table 13, a higher percentage of chairs who served less than two years rated 
“stimulating faculty enthusiasm" as a high priority than chairs with four or more years of 
experience. 


Table 11 
Frequencies and Percentages of Chair Ratings of Administrative Priorities by Carnegie 
Classification (N = 762) 


Carnegie classifications 


Administrative responsibility Associates Bachelors Masters Doctorate 


Priority % n % n % n % 

1. Attending to essential administrative tasks Low 0 0 1 0 11 |2 
Moderate 40 6 12,5 42 {18 | 126] 28, 
High 60 44 88. [190 [82, | 319 | 70, 

2. Fostering good teaching in the department Low 0 fl 2 2 1 21 [5 
Moderate 20 20 40 (84 6 203 | 44 


High 


3. Assisting in securing funding from external sources Low 
Moderate 
High 


4. Leading in establishing and monitoring progress on Low 
annual or biannual department goals Moderate 


n 
0 
4 
6 
0 
2 
8 
7 
3 
0 
0 
6 

High 4 40 {11 22, [93 |AOap | 218 | 48, 
1 
6 
3 
1 
7 
2 
0 
3 
7 
0 
6 
4 


5. Guiding the development of sound procedures for Low 10 (14 28 23 10) | 37 | 8 
assessing faculty performance Moderate 60 28 66 i127 Iss 255 | 56 
High 30 8 16, {83 6, | 167 | 36, 
6. Facilitating successful recruitment and selection of Low 10 B 6 18 8 23/5 
promising faculty Moderate 70 27 (4. |78 (33, | 116 | 25 
High 20 20 40. [137 [59, | 320} 70, 
7. Communicating the department's needs to the dean Low 0 {1 2 3 1 5 1 
and other appropriate administrators Moderate 30 16 bB2 ‘9 Bo 104 | 23 
High 70 {33 (66 {161 (69 350 | 76 
8. Developing collegiality/cooperation among faculty Low 0 £8 6 4 2 13 [3 
members Moderate 60 23 46 [112 48 | 200| 44 


High 
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9. Stimulating research, scholarly activity, and/or Low 4 40 |11 22, 21 9% 12 | 3. 
creative endeavors in the department Moderate | 5 50 32 64, [140 |60, | 201 | 44, 
High 1 10 |7 14. {72 Bln | 246 | 54, 
10. Guiding the development of a sound long-range Low 0 0 |7 14, |11 | 13 | 3p 
plan to carry out departmental programs Moderate | 4 40 28 be 10 47 188 | 41 
High 6 60 (15 BOs {112 48, | 257 | 56, 
11. Promoting a positive image of the department Low 0 0 ff 2 8 3 20 |5 
within the campus community Moderate|7 |70 29 68 [90 9 | 191 | 42 
High |3 |30 20 MO {135 [58 | 247] 54 
12. Fostering the development of each faculty Low 0 0 6 12, |10 [4 17 | 4b 
member's special talents or interests Moderate | 5 50 1 (62a, 131 [56, | 204] 44, 
High 5 50 (13 26, (992 9. | 238 | 52, 
13. Ensuring that new faculty and staff are acquainted Low 0 0 2 4 8 3 28 |6 
with departmental procedures, priorities, and Moderate | 5 50 B5 170, |105 45, | 249 | 54,, 
EXBECIANONS High |5 |50 113 6, [120 52, | 181 | 40, 
14. Clearly communicating expectations of the campus| Low 0 0 £8 6 9 4 15 [3 
administration to the faculty Moderate | 6 60 27 64 hoo 43 222 | 48 
High 4 40 20 40 {124 {53 222 | 48 
15. Stimulating or rejuvenating faculty Low 0 0. 68 16 {18 |8 27 |6 
vitality/enthusiasm Moderate | 8 80 20 bo i136 58 251155 
High 2 20 (17 B4 79 {4 179 | 39 
16. Facilitating curriculum development Low 0 0 8 14. +6 40 |9 
Moderate | 5 50 {31 (62 [128 [55 261 | 57 
High 5 50 {15 BO 90 £9 158 | 34 
17. Establishing trust between himself/herself and Low 0 0 2 4 3 1 6 1 
members of the faculty Moderate |6 | 60 22 44 [69 0 | 127] 28 
High 4 40 26 62 /161 {69 325 | 71 
18. Promoting a positive image of the department to Low 2 20 (6 12 19 |8 27 |6 
off-campus constituencies Moderate |6 | 60 27 64 [100 43 | 213 | 47 
High 2 20 {17 B4 {114 449 218 | 48 
19. Rewarding faculty in accordance with their Low 1 10 10 20, (24 |110a,} 25 | 5b 
contributions to the department Moderate | 8 80 B30 60a» 1134 58, | 210 | 46, 
High 1 10 10 20. |74 2. | 224 | 49, 
20. Ensuring the assessment of student learning Low 0 0 {5 10 {18 {8 38 | 8 
outcomes is meaningful and ongoing Moderate | 5 50 BO 60 22 |53 266 | 58 
High 5 50 {15 BO 92 40 151 | 33 
21. Actively supporting student recruitment and Low 0 0 £B 6 16 /7 45 | 10 
retention efforts Moderate |7 | 70 23 46 (91 9 | 217| 47 
High 3 30 24 48 [125 {54 196 | 43 


Note. Percentages with different subscripts differ, p < .05. 


Table 12 


Frequencies and Percentages of Chair Ratings of Administrative Priorities by Institutional 


Control (N = 762) 
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Institutional control 


Public Private 
Administrative responsibility Priority n % n % 

1. Attending to essential administrative tasks Low 5 la 7 Ob 
Moderate | 155 | 26, 23 16p 

High 444 74 115 79 

2. Fostering good teaching in the department Low 19 3 5 3 
Moderate | 259 | 43 50 34 

High 328 54 90 62 
3. Assisting in securing funding from external sources Low 121 | 20, 89 61p 
Moderate | 285 | 47, 45 31p 

High 200 | 33, 11 8p 

4. Leading in establishing and monitoring progress on annual or Low 21 3 10 7 
biannual department goals Moderate | 322 55 71 49 
High 262 43 64 44 

5. Guiding the development of sound procedures for assessing faculty Low 51 8 24 17 
performance Moderate | 341 55 75 55 
High 215 35 46 32 
6. Facilitating successful recruitment and selection of promising faculty Low 21 35 24 17 
Moderate | 186 31 42 29 
High 400 | 66, 79 345 

7. Communicating the department's needs to the dean and other Low 3 0 6 1 
appropriate administrators Moderate | 159 pre 33 26 
High 445 73 106 73 

8. Developing collegiality/cooperation among faculty members Low 18 3 2 1 
Moderate | 294 | 49, 47 | _ 33 
High 293 | 48, 95 665 
9. Stimulating research, scholarly activity, and/or creative endeavors in Low 30 Da 18 12) 
the department Moderate | 286 | 47, 92] 63, 
High 291 | 48, 35 24p 

10. Guiding the development of a sound long-range plan to carry out Low 18 3 13 9 
departmental programs Moderate | 275 | 45 55 38 
High 313 52 77 33 
11. Promoting a positive image of the department within the campus Low 13 2s 16 11, 
community Moderate | 265 | 44 | 52| 36 
High 328 54 77 53 

12. Fostering the development of each faculty member's special talents Low 23 4 10 7 
or interests Moderate | 307 | 51 64| 44 
High 277 | 46 71 49 

13. Ensuring that new faculty and staff are acquainted with Low 25 4a 13 9p 
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departmental procedures, priorities, and expectations Moderate | 339 56, 55 38, 
High 242 | 40, 77 33p 

14. Clearly communicating expectations of the campus administration Low 17 3a 10 7b 
to the faculty Moderate | 304 | 50, 51 35p 
High 286 | 47, 84 38b 
15. Stimulating or rejuvenating faculty vitality/enthusiasm Low 32 Ba 1 14, 
Moderate | 357 | 59, 63 43p 

High 216 36 61 42 

16. Facilitating curriculum development Low 44 7 14 10 
Moderate | 368 | 61a 97 39p 
High 194 | 32, 74 dlp 

17. Establishing trust between himself/herself and members of the Low 8 1 3 2 
faculty Moderate | 189 | 31 | 35| 24 
High 409 67 107 74 

18. Promoting a positive image of the department to off-campus Low 36 6 18 12 
constituencies Moderate | 287 47 59 Al 
High 283 47 68 47 
19. Rewarding faculty in accordance with their contributions to the Low 36 64 24 17> 
department Moderate | 303 | 50 79 | 54 
High 267 | 44, 42 29p 

20. Ensuring the assessment of student learning outcomes is meaningful} Low 51 8 10 7 
and ongoing Moderate | 359 | 602 64} 44 
High 192 | 32a 71 49 

21. Actively supporting student recruitment and retention efforts Low 45 7 19 13 
Moderate | 279 | 46 59 41 

High 282 47 66 46 


Note. Percentages with different subscripts differ, p < .05. 


Table 13 


Frequencies and Percentages of Chair Ratings of Administrative Priorities by Years of 


Service (N =762) 
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1 year 2-3 years 4-5 years > 5 years 

Administrative responsibility Priority on % n % n  % n % 

1. Attending to essential administrative Low 1 1 4 2 1 1 6 2 
tasks Moderate | 30 26 45 21 33 21| 69 27 
High 84 73 170; 78} 121} 78} 181} 71 

2. Fostering good teaching in the Low D 6 8 4 2 1 7 3 
deparune at Moderate | 33 | 29 | 83] 38] 75| 48] 117| 46 
High 75 65 128) 58 79 51} 113] 52 
3. Assisting in securing funding from Low 45 39 71 32 38 24] 59 23 
extemal sources Moderate | 39 | 34 89} 41| 74] 47| 121] 47 
High 31 27 59 27 44 28 | 77 30 

4. Leading in establishing and monitoring Low 8 7 8 4 4 3 9 4 
progress on annual or biannual department | wioderate | 54 47 110| 50 82 531 148| 58 
a High | 53 | 46 | 101/ 46| 70| 45] 99 | 39 
5. Guiding the development of sound Low 10 9 27 12 15 10; 22 9 
procedures for assessing faculty Moderate | 56 49 106] 48 93 60| 159! 62 
einai Hign | 49 | 43 | 86] 39] 48| 31/ 77 | 30 
6. Facilitating successful recruitment and Low 9 8 15 7 7 4 11 4 
selection of promising faculty Moderate | 35 30 70 Ew 53 34| 71 28 
High 71 62 134; 61 96 62| 176] 68 

7. Communicating the department's needs Low 2 2 1 0 1 1 5 2 
to the dean and other appropriate Moderate | 26 | 23 57/ 26; 39| 25] 71 | 28 
ap ears High | 87 | 76 | 161/ 74| 116] 74| 182| 71 
8. Developing collegiality/cooperation Low 3 3 6 3 3 2 8 3 
among faculty members Moderate | 44 38 87 40 82 63. |' 733 |. 59 
High 68 59 124; 57 70 45| 117 | 45 

9. Stimulating research, scholarly activity, Low 11 10 16 7 8 5 12 5 
and/or creative endeavors in the Moderate | 56 49 113) S59 77 49| 128) 50 
VEE wes Hign | 48 | 42 | 90] 41] 71| 46/ 118] 46 
10. Guiding the development of a sound Low 10 9 11 5 3 2 ) 2 
long-range plan to carry out departmental | \j4oderate 44 38 86 39 76 49| 123) 48 
Pee High | 61 | 53 | 122| 56| 77| 49| 129| 50 
11. Promoting a positive image of the Low 5 4 10 5 2 1 11 4 
department within the campus community | yyoderate 4S 39 88 40 68 44} 115| 45 
High 65 57 121; 55 86 55} 131] 51 

12. Fostering the development of each Low 6 5 14 6 5 3 10 4 
faculty member's special talents or interests} Wogerate | 63 55 100! 46 77 49| 1291 50 
High 46 40 105; 48 74 47| 119 | 46 

13. Ensuring that new faculty and staff are Low 9 8 14 6 6 4 9 2 
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acquainted with departmental procedures, | Moderate | 56 49 111 51 81 52] 148 57 
priorities, and expectations High 50 43 93 43 69 44| 1011 39 

14. Clearly communicating expectations of Low 6 5 7 3 4 3 10 4 
the campus administration to the faculty Moderate | 38 33 101! 46 76 49| 142| 55 
High 71 62 111} 51 76 49| 106} 41 

15. Stimulating or rejuvenating faculty Low 8 7 18 8 9 6 19 7 
vitality/enthusiasm Moderate | 46 AO, 116] 53.4 98 636), 461. | ea, 
High 61 53a 85 | 39,4 49 31h} 76 | 30, 

16. Facilitating curriculum development Low 9 8 18 8 15 10; 16 6 
Moderate | 60 52 118; 54 84 54} 160|/ 62 

High 46 40 83 38 56 36} 82 32 

17. Establishing trust between Low 4 3 3 1 1 1 4 2 
himself/herself and members of the faculty | \yoderate 23 20 63 29 64 41| 74 29 
High 88 77 153] 70 91 58} 179| 70 

18. Promoting a positive image of the Low 10 9 22 10 9 6 13 5 
department to off-campus constituencies | ioderate | 50 43 94 43 74 46| 129| 50 
High eye) 48 103} 47 76 49} 115} 45 

19. Rewarding faculty in accordance with Low 11 10 20 9 11 vA 18 7 
their contributions to the department Moderate | 55 48 105| 48 77 50! 142 55 
High 49 43 94 43 67 43 | 98 38 

20. Ensuring the assessment of student Low 10 9 24 11 11 7 16 6 
learning outcomes is meaningful and Moderate | 60 53 110! 50 88 67 | 464. 64 

ongoin 

ene High 44 39 84 39 55 36 | 77 30 
21. Actively supporting student Low 15 13 13 6 11 7 25 10 
recruitment and retention efforts Moderate | 38 aa 99 45 72 46| 131 51 
High 61 54 107; 49 72 46}; 102 | 40 


Note. Percentages with different subscripts differ, p < .05. 


Group Comparisons in Chair Self-Ratings of Performance 


Administrative responsibilities 


To examine subgroup differences in chair self-ratings of performance on administrative 
responsibilities, we conducted a series of two-way ANOVAs on each of the chair means. The 


independent variables were institutional control (public vs. private) and number of years the 


chair had served (1 year, 2-3 years, 4-5 years, > 5 years). All main effects and two-way 


interactions were examined. Because of the large number of ANOVAs performed, the Type I 
error rate was set at .0024 for the 21 responsibilities. We found no significant two-way 


interactions, but we did find a few main effects. With respect to institutional control, chairs from 


public institutions rated themselves (M = 3.30, SD = 1.11) higher than chairs at private 
institutions (M = 2.60, SD = 1.20) for the item “assisting in securing funding from external 


sources”, F (1, 574) = 24.35, p < .001, 77° = .04. We also found two differences by chair years of 
service. Chairs having served one year or less reported lower self-ratings in “recruiting 


promising faculty” (M = 3.83, SD = 0.93) than chairs with three-to-four years of service (M = 
4.17, SD = 0.81) and chairs with greater than five years of service (M = 4.19, SD = 0.93), F(1, 
574) = 5.52, p= .001, 7° = .03. Chairs with two-to-three years of service (M = 3.60, SD = 0.90) 
rated themselves lower than chairs with greater than five years of service (M = 3.93, SD = 0.84) 
for the item “stimulating research,” F(1, 574) = 6.87, p < .001, 77°= .03. 


Personal characteristics and administrative methods 


ANOVAs were also conducted on chair self-ratings of the 11 personal characteristics by 
institutional control and years of service. Type I error was set at .05/11 = .005. There were no 
significant two-way interactions or main effects. Similarly, we conducted the same procedures 
for chair self-ratings on the 21 administrative methods. Types I error was set at .05/21 = .0024. 
Again, we found no significant two-way interactions or main effects. 


Next, we conducted a series of ANOVAs for chair self-ratings of performance on each of 
the 21 administrative responsibilities by Carmegie classification. We did not include associate- 
level institutions due to small sample size (n = 10). With a Bonferroni adjustment, each ANOVA 
was conducted at a = .05/21 = .0024. As shown in Table 14, we found several differences in 
chair self-ratings of performance by Carnegie classification. Tukey post-hoc tests showed that 
chairs in doctoral institutions rated themselves higher on the following three administrative 
responsibilities than chairs at master’s and baccalaureate institutions: securing external funding, 
stimulating research, and rewarding faculty. For the item “facilitating curriculum development” 
chairs at master’s institutions rated themselves higher than chairs at doctoral and baccalaureate 
institutions. 


Table 14 
Descriptive Statistics and ANOVA Results for Chair Self-ratings of Performance on 
Administrative Responsibilities by Carnegie Classification 


Associates Bachelors Masters Doctoral Total 
(n= 10) (n = 50) ne rst (n = 455) (n = 743) 
Admin. Responsibility 


1. Attending to essential 
administrative tasks 4.10 4.20 | 0.62 | 4.23 | 0.61 
2. Fostering good teaching in 


the department 

3. Assisting in securing 

funding from external sources} 2.83 | 1.47 

4. Leading in establishing and 

monitoring progress on 

annual or biannual 

department goals 3.70 | 0.68 3.69 | 0.81 | 3.64 | 0.83 
salons 


5. Guiding the development 
of sound procedures for 
assessing faculty performancef 3.56 | 0.73 
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6. Facilitating successful 
recruitment and selection of 
promising faculty 3.20 


7. Communicating the 
department's needs to the 4.30 | 0.48 § 4.41 | 0. 
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dean and other appropriate 
administrators 


8. Developing 
collegiality/cooperation 
among faculty members al 0.88 | 3.94 | 0.91 


9. Stimulating research, 
scholarly activity, and/or 
creative endeavors in the 
department 

10. Guiding the development 
of a sound long-range plan to 
carry out departmental 
programs 


11. Promoting a positive 
image of the department 
within the campus 
community 


lod 0.81 sos ose los 
ot 
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3.22, 045 


3.53 


of 


.008 


os 
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0 
12. Fostering the 
development of each faculty 
member's special talents or 
interests 

13. Ensuring that new faculty 
and staff are acquainted with 
departmental procedures, 
priorities, and expectations 
14. Clearly communicating 
expectations of the campus 


administration to the faculty | 3.90 | 0.74 
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.008 
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18. Promoting a positive 
off-campus constituencies 3.89 4.10 | 0.84 | 4.06 | 0.85 
retention efforts 3.40 | 0.52 | 4.04 | 0.82 [| 4.00 | 0.91 | 3.91 | 0.89 | 3.94 | 0.89 §.1.11 | .329 | .003 


15. Stimulating or 
rejuvenating faculty 
vitality/enthusiasm 3.50 
16. Facilitating curriculum 
development 4.20 0.98 | 3.97, | 0.87 § 3.76, | 0.87 | 3.82 | 0.88 | 6.49 | .002 
19. Rewarding faculty in 
accordance with their 
contributions to the 
department 3.00 | 0.94 J 3.21, 3. a 0.92 93.75, | 0.93 | 3.63 | 0.94 | 10.58 
20. Ensuring the assessment 
of student learning outcomes 
is meaningful and ongoing 3.70 | 0.82 8 
21. Actively supporting 
Note. One-way ANOVAs were conducted to test whether ratings of importance varied between bachelors, 
master’s, and doctoral institutions. Post-hoc tests were conducted only following a significant omnibus F (p < 
.0017 correction because of the numerous ANOVAs conducted). Means with different subscripts differ, p < .05. * 


17. Establishing trust between 
himself/herself and members 
of the faculty 4.20 | 0.63 | 4.00 | 0.83 4.12 | 0.77 | 4.15 | 0.77 
image of the department to nee 
student recruitment and 
df = 2, 730 
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Group Comparisons in Faculty Ratings of the Chair’s Performance 


Administrative responsibilities 


Comparisons were also made among subgroups of faculty in ratings of the chair’s 
performance. For the 21 administrative responsibilities, raters used the same five-point scale 
their chair employed for self-ratings, but they were given the additional option of Omit Response. 
To examine the possibility of subgroup differences in ratings of administrative responsibilities, 
we conducted a series of two-way ANOVAs on each of the faculty aggregated means. The 
independent variables were institutional control (public vs. private) and chair experience (1 year, 
2-3 years, 4-5 years, > 5 years). With the Bonferroni adjustment applied to Type I error, 
respectively, for the administrative responsibilities, personal characteristics, and administrative 
methods, we found no significant two-way interactions or main effects. 


Next, we examined whether faculty ratings of administrative responsibilities differed by 
Carnegie classification. To check whether department size might have been a confounding 
variable in the comparisons made by Carnegie classification, we performed an ANOVA (a = .05) 
on the number of faculty in each department. Departments from doctoral institutions (M = 22.63, 
SD = 22.64) tended to have more faculty members (M = 11.70, SD = 5.42) than those from 
bachelors and master’s (M = 16.03, SD = 8.66) institutions, F(2,739) = 11.64, p < .001, 7? = .03; 
however, the effect size was minimal. 


We then conducted a series of one-way ANCOVAs on the average faculty ratings of 
chair administrative performance by Carnegie classification and included department size as a 
covariate. Table 15 presents unadjusted means and standard deviations. There were several 
Statistically significant differences by Carnegie classification (see Table 16 for adjusted means 
and standard errors). Bonferroni post-hoc tests revealed a consistent pattern, showing higher 
average ratings for chairs in master’s than doctoral institutions on 12 of 21 responsibilities. The 
greatest differences were seen in the following items: actively supporting student recruitment and 
retention efforts, attending to essential administrative tasks, and guiding the development of 
sound procedures for assessing faculty performance. 


Table 15 
Descriptive Statistics for Faculty Ratings of Administrative Responsibilities by Carnegie 
Classification 


Associates Bachelors Masters Doctoral Total 
(n= 10) (n= 50) (n = 230) (n = 455) (n = 743) 
Administrative Responsibility 


S 
wn 
NI 


1. Attending to essential administrative tasks 0.27 | 4.29 | 0.50 0.53 0.59 | 4.23 
1 1 


2. Fostering good teaching in the department 


B. Assisting in securing funding from external Pees ee 
Sources 4.14 | 0.64 
4. Leading in establishing and monitoring peta fa 
progress on annual or biannual department goals 0.37 


6. Guiding the development of sound procedures 
for assessing faculty performance 4.33 | 0.35 


6. Facilitating successful recruitment and selection 
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of promising faculty aes eee (eee ee eee ee ee ee ee es) 
7. Communicating the department's needs to the 

dean and other appropriate administrators Pear 
8. Developing collegiality/cooperation among 

9. Stimulating research, scholarly activity, and/or 

10. Guiding the development of a sound long- 

11. Promoting a positive image of the department 

within the campus community ra a es oa 4.30 | 0.55 al ie 64 a 22 


12. Fostering the development of each faculty 
member's special talents or interests 


13. Ensuring that new faculty and staff are 

acquainted with departmental procedures, 

priorities, and expectations 4.04 0.62 

14. Clearly communicating expectations of the pala rae Aiea 

campus administration to the faculty Meal 33 4.23 4.02 | 0.59 | 4.09 ee 58 
15. Stimulating or rejuvenating faculty 2 e2. oneal eres Eaesaleeal 3.6 | 07 
Vitality/enthusiasm 4.10 | 0.43 J 3.72 | 0.78 | 3.79 3.59 | 0.75 | 3.66 | 0.75 


16. Facilitating curriculum development | 4.07 | 0.58 | 3.88 | 0.62 | 
17. Establishing trust between himself/herself and aalead bea eat lcci! Rae prea 
members of the faculty 4.39 | 0.44 | 3.94 | 0.82 

18. Promoting a positive image of the department Pars Pre eesciceal elec A 
to off-campus constituencies 4.54 | 0.26 | 4.19 | 0.64 

19. Rewarding faculty in accordance with their lessee nad bese ieee pt. peo ale 
contributions to the department 4.04 | 0.54 § 3.70 | 0.70 J 3.83 | 0.65 

20. Ensuring the assessment of student learning 

21. Actively supporting student recruitment and 

retention efforts 4.36 | 0.27 | 4.26 | 0.48 § 4.28 | 0.54 | 4.06 | 0.6 4.15 | 0.5 


ee) 


Table 16 
Adjusted Means and ANCOVA Results for Faculty Ratings of Administrative Responsibilities by 


Carnegie Classification 
Bachelors Masters Doctoral 


(n= 50) (n = 230) (n = 455) Omnibus Statistics 
[Administrative Responsibility M M 
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1. Attending to essential administrative tasks 

2. Fostering good teaching in the department 

B. Assisting in securing funding from external sources 
4. Leading in establishing and monitoring progress on 
annual or biannual department goals 

5. Guiding the development of sound procedures for 
assessing faculty performance 

6. Facilitating successful recruitment and selection of 
promising faculty 

7. Communicating the department's needs to the dean 
land other appropriate administrators 

8. Developing collegiality/cooperation among faculty 
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9. Stimulating research, scholarly activity, and/or 0.10 : . 2.94 § .054 § .008 
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creative endeavors in the department = ln aS er aaa 
10. Guiding the development of a sound long-range fecal ra 

plan to carry out departmental programs 3.99 0.11 J 4.02, | 0.05 3.814 | 0.03 8.51 
11. Promoting a positive image of the department [aera lee lal reed war 
within the campus community 4.21 0.10 | 4.31 | 0.04 4.16 | 0.03 4.54 
12. Fostering the development of each faculty asl a ee 
member's special talents or interests 3.98 0.11 | 4.01, | 0.04 3.814 


13. Ensuring that new faculty and staff are acquainted 
with departmental procedures, priorities, and 
expectations 4.06 0.10 | 4.13, | 0.04 3.95p | 0.03 


14. Clearly communicating expectations of the rea fees 

campus administration to the faculty 4.06 4.22, | 0.04 4.03, | 0.03 9.50 
15. Stimulating or rejuvenating faculty 

Vitality/enthusiasm 3.73 6.41 
16. Facilitating curriculum development 4.00 | 0.10 0.04 0.03 | 9.31 
17. Establishing trust between himself/herself and 

members of the faculty 3.93 0.13 | 4.04 | 0.05 3.86 | 0.04 4.54 
18. Promoting a positive image of the department to 

off-campus constituencies 4.22 
19. Rewarding faculty in accordance with their 

contributions to the department 3.68 
20. Ensuring the assessment of student learning lease era fs bala 
loutcomes is meaningful and ongoing 4.05 4.14, | 0.04 3.914 12.10 
21. Actively supporting student recruitment and 

retention efforts 4.22 0.09 | 4.30, | 0.04 4.06, | 0.03 14.39 
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< .001 


037 


Note. One-way ANCOVAs were conducted to test whether ratings of importance varied between bachelors, 
master’s, and doctoral institutions. Post-hoc tests were conducted only following a significant omnibus F (p < 
.0017 correction because of the numerous ANCOVAs conducted). Means with different subscripts differ, p < .05. 
* df = 2, 739 


Personal characteristics and administrative methods 


To examine subgroup differences in ratings of personal characteristics, we conducted a 
series of two-way ANOVAs on each of the faculty aggregated means. The independent variables 
were again institutional control and number of years the chair had served. For each of the 11 
personal characteristics, Type I error rate was set at .05/11 = .005. As before, faculty rated the 
chair with the same five-point scale the chair used to complete self-ratings with the additional 
option of Omit Response. There were no significant two-way interactions or main effects. 
Similarly, we performed the same analyses on the 21 administrative methods, using a reduced 
a of .05/21 = .0023. As with ratings of personal characteristics, faculty responded with the same 
five-point scale chairs used along with the additional Omit Response option. Again, we found no 
significant two-way interactions or main effects. Table 17 presents means and standard 
deviations. 
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Table 17 
Descriptive Statistics for Faculty Ratings of Personal Characteristics by Carnegie Classification 
Associates Bachelors Masters Doctoral Total 
(n = 10) (n = 50) (n= 230) | (n= 455) (n = 743) 
Personal Characteristics SD M 
22. Problem solving ability 4.17 
23. Demonstrates caring 4.15 | 4.17 | 0.66 | 
24, Practical judgment 4.16 
25. Trustworthy 4.20 
26. Flexibility/adaptability 417 | 069 | 417 | 064] 4.02 | 066] 408 | 0.66 | 
27. Fairness 4.14 | 4.05 | 0.66 | 
28. Organizational skills 4.09 | 069 | 4.20 | 0.59 | 
29. Consistency 4.11 
30. Enterprising 411 | 061 | 412 | 0.60 | 
31, Institution-centered 4.30 
32. Clarity 4.08 | 3.95 | 0.66 | 


Next, we conducted ANCOVAs on the average faculty ratings of 11 personal 
characteristics by Carnegie classification with department size as the covariate. There were 
statistically significant differences by Carnegie classification on seven of 11 items (see Table 
18). Bonferroni post-hoc tests revealed chairs from master’s institutions were rated higher than 
chairs from doctoral institutions. The greatest differences were found in the following items: 
institution-centered, consistency, and demonstrates caring. 


Table 18 


Adjusted Marginal Means and ANCOVA Results for Faculty Ratings of Personal Characteristics 


by Carnegie Classification 


Personal Characteristics 
22. Problem solving ability 
23. Demonstrates caring 
24, Practical judgment 

25. Trustworthy 

26. Flexibility/adaptability 
27, Fairness 

28. Organizational skills 
29. Consistency 

30. Enterprising 

31. Institution-centered 
32. Clarity 


Bachelors Masters Doctoral 

(n = 50) (n = 233) (n = 460) Omnibus Statistics 
| oT 
| 6.00 | 
| .006 | 
6.52 | .002 | .017 
5.17 | .006 | .014 
8.80 023 

4.05 0.10 4.13, 0.04 | 3.95, 0.03 | 6.58 .001 | .018 


Note. One-way ANCOVAs were conducted to test whether ratings of importance varied between bachelors, 
master’s, and doctoral institutions. Post-hoc tests were conducted only following a significant omnibus F (p < .005 
correction because of the numerous ANCOVAs conducted). Means with different subscripts differ, p <.05. * df= 


2, 73 


We conducted the same procedures on the 21 administrative methods. Table 19 presents 
means and standard deviations. We found numerous statistically significant differences by 
Carnegie classification (see Table 20). Bonferroni post-hoc tests revealed a consistent pattern of 
differences: Chairs in institutions awarding masters degrees were rated higher on 17 of 21 
administrative methods than those awarding doctoral degrees. The greatest differences were 
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observed in the following items: facilitating positive relationships, allocating responsibilities, 
putting suggestions into action, making sure his/her part is understood, and letting faculty know 
expectations. 


Table 19 
Descriptive Statistics for Faculty Ratings of Administrative Methods by Carnegie Classification 


Associates Bachelors Masters Doctoral ee 
(n= 10) (n = 50) (n = 230) (n = 455) (n = 743) 


Administrative Method 
33. Allocates responsibilities 4.32 | 0.35 | 4.00 | 061] 4.06 |o60 | 3.87 [061 | 3.94 | 0.61 
34, Resolves conflicts 
35. Assists faculty goals 4.24 | 0.40 | 3.96 0.60 0.64 | 3.87 [0.63 | 3.92 | 0.63 
36. Makes sound suggestions 0.27 | 4.12 
37. Advocates dept. to admin 4.18 [0.60_| 
38. Acts as though morale vital [405 [0.56 
39. Tries out new ideas 
40. Sees that fac. are working 056 | 3.82 1059 | 3.88 10.58 
41. Looks out for fac. welfare | 3.93 | 0.66 | 
42. Lets fac. know expectations = ‘00 0.57 
43. Promotes inclusiveness and diversity 0. 57 0.56 
44. Coordinates fac. work [406 | 060) 0.62 3. ‘OI 0.62 
45, Explains the basis for decisions rar fase fs 
46. Lets members know good job 
47. Makes sure his/her part is understood [407 | 066. 0.62 
48. Acts as though accomplishments vital 


_ Maintains standards of performance (anne aa 
. Puts suggestions into action 
. Facilitates positive relationships 


. Encourages teamwork 
- Provides feedback 4.30 [0.39 | 410 | 050] 415 [058 | 398 [061 | 4.05 [060 


Table 20 
Adjusted Marginal Means and ANCOVA Results for Faculty Ratings of Administrative Methods 
by Carnegie Classification 
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. Makes sure his/her part is understood 4.18, 


. Acts as though accomplishments vital] 4.19 | 0.09 | | 

. Maintains standards of performance | 4.21, | 0.04 | 4.015 

. Puts suggestions into action 

. Facilitates positive relationships 

. Encourages teamwork | 4.18. | 0.04 | 3.975 
Provides feedback 4.15 0.10 | 4.16, | 0.04 3.98 
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Bachelors Masters Doctoral 
(n= 50) (n = 233) | (n= 460) 
Administrative Method | M | SE | M 
33. Allocates responsibilities 0.10 
34. Resolves conflicts 
35. Assists faculty goals 
36. Makes sound suggestions 
37. Advocates dept. to admin 
38. Acts as though morale vital 4.06, 3.82, | 0.03 
39. Tries out new ideas | 4.12, | 0.04 | 3.93» 
40. Sees that fac. are working 0.04} 3.82, | 0.03 
41. Looks out for fac. welfare 
42. Lets fac. know expectations | 4.07 | 0.09 | 
43. Promotes inclusiveness and diversity | 4.19 | 0.09 | | 4.32. | 0.04 | 4.16p 
44. Coordinates fac. work 3.815 
45. Explains the basis for decisions 3.98 
46. Lets members know good job | 4.13 | 0.09 | 
0.04 
0.04 
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Note. One-way ANCOVAs were conducted to test whether ratings of importance varied between bachelors, 
master’s, and doctoral institutions. Post-hoc tests were conducted only following a significant omnibus F (p < .0017 
correction because of the numerous ANCOVAs conducted). Means with different subscripts differ, p < .05. * df= 2, 
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Summary judgments 


We also conducted a one-way MANCOVA for faculty ratings of the two summary 

judgment items ("I have confidence in the chair's ability to provide future leadership to the 

department"; "Overall, this chair has provided excellent leadership") by Carnegie classification 
(Table 21 presents means and standard deviations). Response options ranged from 1 = Strongly 
Disagree to 5 = Strongly Agree. The multivariate test was significant, F(2, 738) = 3.38, p = .001, 
n° = .01, as well as both univariate tests (see Table 22). Bonferroni post-hoc tests revealed that 
chairs in institutions awarding masters degrees were rated higher than those awarding doctoral 
degrees on both summary judgment items. 
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Table 21 
Descriptive Statistics for Faculty Ratings of Summary Judgment Items by Carnegie 
Classification 


Associates Bachelors Masters Doctoral 
(n = 230) (n = 455) 


Administrative Method 


54. Confidence in leadership 
55. Excellent Leadership 4.51 


Table 22 
Adjusted Means and MANCOVA Results for Faculty Ratings of Summary Judgment Items by 
Carnegie Classification 
Bachelors Masters Doctoral 
|. - d (n= 50) (n = 233) (n= 460) [Omnibus Statistics 
| Administrative Method _| 


Administrative Method SE SE | Ex] p | 


| SE | 
0.03 | 6.66] .001] 018 _| 
4.19, | 0.05 


Note. One-way MANCOVA was conducted to test whether ratings of importance varied between bachelors, 
master’s, and doctoral institutions. Post-hoc tests were conducted only following a significant omnibus F (p < .05 
correction because of the numerous ANOVAs conducted). Means with different subscripts differ, p < .05. * df= 2, 
738 


Summary 


Respondents represented a diverse group of departments that varied in size, region of the 
country, Carnegie classification, and institutional control. The chairs were a fairly experienced 
group of administrators, the majority of whom had some courses being taught by adjuncts. 
Although most participants served institutions offering either the master’s or doctoral degree, 
individuals from associate and bachelors institutions were also included in the sample. 


There were several statistically significant differences in chair prioritization of 
administrative responsibilities. Differences were found between public and private institutions 
and among Carnegie classifications. Possible explanations for the observed differences in 
priorities are the contrasting missions of public and private institutions and the variance in 
research and teaching loads associated with Carnegie classifications (e.g., doctoral institutions 
typically are more research intensive). For example, responsibilities related to external funding 
and stimulating research were of greater importance to chairs at doctoral institutions than chairs 
at master's or bachelor's institutions. Such differences support the validity of having chairs rate 
the importance of administrative responsibilities. 


Similarly, we found differences in faculty ratings of chair performance by Carnegie 
classification. Chairs from master’s institutions were rated higher than those at the doctoral level 
on more than half the responsibilities and most of the personal characteristics and administrative 
methods. One explanation might be that chairs in research-intensive institutions have a greater 
number of responsibilities to manage and more faculty members to serve than chairs at less 
research-oriented institutions. Descriptively, a greater proportion of chairs from doctoral 
institutions rated 13 of 21 administrative responsibilities as either a high or medium priority than 
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chairs at either master's or bachelor's institutions. Tests of statistical significance showed that 
chairs from doctoral institutions assigned higher priorities to five responsibilities and had 
significantly larger departments than those from masters and baccalaureate institutions. Trying to 
juggle so many priorities and serve so many faculty may have lowered their performance ratings 
relative to chairs without responsibilities associated with a strong research orientation (e.g. 
securing external funding, stimulating research). 


VALIDITY 


Validity concerns evidence that supports the interpretation and use of scores obtained 
from an assessment. Evidence presented in this section, therefore, addresses whether the IDEA FSC 
can be used for its intended purpose. The IDEA system is primarily intended for formative or 
developmental purposes. Consequently, it is designed to measure (a) faculty perceptions of the 
chair’s effectiveness and (b) the congruence between the chair’s self-ratings and faculty ratings of 
chair performance (i.e., gap analysis). The Chair Report presents this information along with 
descriptions of supporting personal characteristics and administrative methods associated with 
high performance of each responsibility. If institutions choose to use the report for summative 
purposes, IDEA strongly recommends that additional indicators of effectiveness the institution 
deems appropriate also be considered. 


Evidence of Content Validity 


Evidence for content validity, which concerns the development of the wording and format 
of items in a survey, can be found in IDEA Technical Report No. 14. 


Validity Evidence Based on Internal Structure 


Evidence for internal structure indicates the degree to which relationships among 
responses to items conform to an intended structure. In this section, we report on the internal 
structure of chair priority ratings of administrative responsibilities, chair self-ratings of 
performance, as well as faculty ratings of the chair's performance of administrative 
responsibilities, personal characteristics, and administrative methods. Then, we provide evidence 
that faculty ratings of chair characteristics and administrative methods are related to ratings of (a) 
summary judgments, and (b) administrative responsibilities in predictable patterns. 


Exploratory Factor Analysis of Chair Priority Ratings 


In a previous exploratory factor analysis, Benton et al. (2010) found a five-factor solution 
in 474 chair priority ratings of the 20 responsibilities on the Chair Information Form: Personnel 
Management/Development, Developing Positive Climate, Administrative Support, Program 
Leadership, and Building Image/Reputation. In the current CSA, one of the 20 responsibilities 
was removed and two new ones were added, bringing the total number to 21. To examine the 
underlying structure of chair ratings on those 21 responsibilities, we performed principle 
components analysis (PCA) (not forcing the number of factors extracted) on chair priority 
ratings. Table 23 presents component matrix coefficients along with factor eigenvalues and 
percent of variance explained following the varimax rotation. The factor structure for the five- 
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factor solution was very similar to that reported by Benton et al. (2010), except that Item 5 
loaded on a different factor and two new items (20 and 21) loaded on separate factors. The first 
factor, Program Leadership, contained five items (2, 5, 13, 16, 20) that concern assessment, 
curriculum, and teaching. The second factor, which contained six items (3, 4, 6, 9, 10, 19), 
related to Personnel Management, refer to fostering research, external funding, and 
recruitment/retention of quality faculty. Four items (8, 12, 15, 17) pertained to a third factor, 
Developing Positive Climate, by encouraging collegiality, trust, and faculty development. A 
fourth factor contained three items (11, 18, 21) connected to Building Image/Reputation both on 
and off campus. Lastly, a factor made up of three items (1, 7, 14), was associated with 
Administrative Support, which concerned communicating department needs and administrative 
expectations. 


Table 23 
Component Matrix Coefficients for Chair Priority Ratings of Administrative Responsibilities 
Following Varimax Rotation (N = 762). 


Factors 
CSA Item 1 2 3 4 S) 
20. Ensuring the assessment of student learning 
outcomes is meaningful and ongoing 726 =©.067 036 .264 ~~=.065 
16. Facilitating curriculum development 715 «6.044 «=©=.069— «1111S 071 
2. Fostering good teaching in the department 664 029 313 -.008 .015 
13. Ensuring that new faculty and staff are acquainted 
with departmental procedures, priorities, and 
expectations 486 178 318 06 = .427 
5. Guiding the development of sound procedures for 
assessing faculty performance 435 339 174 .041  .345 
3. Assisting in securing funding from external sources -014 .708 002 .202 -.164 
9. Stimulating research, scholarly activity, and/or 
creative endeavors in the department 002.681) 9 =.245 = =.089_—s-.005 
6. Facilitating successful recruitment and selection of 
promising faculty 1110 5560 217) -.219 329 
19. Rewarding faculty in accordance with their 
contributions to the department 174 =©.495 367 170 .232 
10. Guiding the development of a sound long-range plan 
to carry out departmental programs 305 §6©.450 -.168 .436 8.094 
4. Leading in establishing and monitoring progress on 
annual or biannual department goals 341 .412 -.0096 400 .211 
8. Developing collegiality/cooperation among faculty 
members 145 021 .754 154 = .094 
17. Establishing trust between himself/herself and 
members of the faculty 102. .089 677 .067 = .282 
12. Fostering the development of each faculty member's 
special talents or interests 215 .408 6532 131 = -.022 


15. Stimulating or rejuvenating faculty 
vitality/enthusiasm 143 .295 531 8.368) = -.052 
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11. Promoting a positive image of the department within 


the campus community 0066 055 .260 .714 ~~ = .194 
18. Promoting a positive image of the department to off- 

campus constituencies 143 162 216 =«.681)— 053 
21. Supporting student recruitment and retention 440 012 135 .462 ~ «2146 
7. Communicating the department's needs to the dean 

and other appropriate administrators -109 .252 0 «=.027)—S 256 ~— «6833 
1. Attending to essential administrative tasks (e.g., class 

scheduling, staffing, finances/budgeting, facilities) 181 -.254 086 .008 .660 
14. Clearly communicating expectations of the campus 

administration to the faculty 11.048 = .192 270 ~—-«.499 
Eigenvalues after rotation 2.615 2.467 2.285 2.067 1.857 
Percent of variance explained 12.45 11.75 1088 9.84 8.84 


Note. Number of factors were unrestricted. Boldface indicates highest factor loadings. Factor 1 = 
Program Leadership/Support; Factor 2 = Personnel Management; Factor 3 = Developing Positive 
Climate; Factor 4 = Building Image/Reputation; Factor 5 = Administrative Support/Leadership. 


Exploratory Factor Analysis of Chair Self-Ratings of Administrative Responsibilities 


We also performed a PCA on the chair self-ratings of the 21 administrative 
responsibilities (not forcing the number of factors extracted). Table 24 presents component 
matrix coefficients along with factor eigenvalues and percent of variance explained following the 
varimax rotation. The factor structure for the five-factor solution was very similar to the chair 
priority ratings of administrative responsibilities except that four items loaded on a different 
factor. Items 4 and 10 loaded on the Program Leadership/Support factor for chair self-ratings of 
performance, whereas they loaded on the Personnel Management factor for chair priority ratings 
of administrative responsibilities. Item 13 loaded on the Program Leadership/Support factor for 
chair priority ratings of administrative responsibilities, but it loaded on the Developing Positive 
Climate factor for chair self-ratings of performance. Lastly, Item 7 loaded on the Administrative 
Support/Leadership factor for chair priority ratings, although it loaded on the Building 
Image/Reputation factor for chair self-ratings of performance. 


Table 24 
Component Matrix Coefficients for Chair Self-Ratings of Administrative Responsibilities 
Following Varimax Rotation (N = 602). 


Factors 
CSA Item 1 2 3 4 5 
10. Guiding the development of a sound long-range plan 
to carry out departmental programs* .707 = =.189) -.093)—s 258 ~—— 216 
20. Ensuring the assessment of student learning 
outcomes is meaningful and ongoing 690 105 393 .093 -.007 
4. Leading in establishing and monitoring progress on 
annual or biannual department goals* 687 185 -012 .263 «144 
16. Facilitating curriculum development 642 101 380 86.053 011 


2. Fostering good teaching in the department 622 249 301 .110~ 143 


ob 


5. Guiding the development of sound procedures for 


assessing faculty performance 2ol 135) 344-028 — 278 
8. Developing collegiality/cooperation among faculty 

members 136 .846 107 128 .101 
17. Establishing trust between himself/herself and 

members of the faculty 146 .768 194 .184  .061 
15. Stimulating or rejuvenating faculty 

vitality/enthusiasm 299 .689 079 .119  .309 
12. Fostering the development of each faculty member's 

special talents or interests 368 .475 107 .167 ~ .374 
1. Attends to admin. details 113 047 .708 = = .133 121 


13. Ensuring that new faculty and staff are acquainted 
with departmental procedures, priorities, and 


expectations* 311 6.166) =6.587)—S 089-328 
14. Clearly communicating expectations of the campus 

administration to the faculty 263 325 «©6531 215 -.005 
18. Promoting a positive image of the department to off- 

campus constituencies 178 = =.167) = 118) 6777'—S 144 
11. Promoting a positive image of the department within 

the campus community 140 .261 226 © .752~ ~=—.044 
21. Supporting student recruitment and retention 316 059 428 .459 = .079 
7. Communicating the department's needs to the dean 

and other appropriate administrators* 0029 .207. 345) 395.233 
6. Facilitating successful recruitment and selection of 

promising faculty -049 158 .288 .055  .684 
9. Stimulating research, scholarly activity, and/or 

creative endeavors in the department 402 .286 002 175  .580 
3. Assisting in securing funding from external sources 145 -.055 -.132 533 8 .565 
19. Rewarding faculty in accordance with their 

contributions to the department 322.164.219.123 543 
Eigenvalues after rotation 3.434 2.593 2.227 2.193 2.037 
Percent of variance explained 16.35 12.35 10.61 10.45 9.70 


Note. Number of factors were unrestricted. Boldface indicates highest factor loadings. Factor 1 = 
Program Leadership/Support; Factor 2 = Personnel Management; Factor 3 = Developing Positive 
Climate; Factor 4 = Building Image/Reputation; Factor 5 = Administrative Support/Leadership. 
* Factor loading differed by Chair importance PCA and Chair self-performance PCA 


Exploratory Factor Analysis of Chair Self-Ratings of Personal Characteristics 


We also performed PCA (not forcing the number of factors extracted) on chair self- 
ratings of personal characteristics. As shown in Table 25, the unrestricted principal component 
analysis revealed a two-factor solution. One factor contained seven characteristics related to 
Faculty and Administrative Management and the other was comprised of four characteristics that 
support Faculty Relationships. 


Table 25 


Component Matrix Coefficients for Chair Self-Ratings of Personal Characteristics Following 


Varimax Rotation (N = 750). 


Factors 
CSA Item 1 2 
28. Organizational skills .767 = -.088 
29. Consistency 751.158 
22. Problem solving ability 664 .160 
24. Practical judgment 633 = 308 
32. Clarity 614 = .299 
30. Enterprising 036 = 241 
31. Institution-centered 446 = .331 
23. Demonstrates caring -.076 815 
26. Flexibility/adaptability 219 .647 
25. Trustworthy 398 ~=-.641 
27. Fairness 482.532 
Eigenvalues after rotation 3.303 2.187 
Percent of variance explained 30.02 19.88 
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Note. Number of factors were unrestricted. Boldface indicates highest factor loadings. Factor 1 = 


Faculty and Administrative Management; Factor 2 = Faculty Relationships 


Exploratory Factor Analysis of Chair Self-Ratings of Administrative Methods 


We also applied PCA to chair self-ratings of the 21 administrative methods. The 
component matrix coefficients and associated statistics presented in Table 26 support a two- 
factor solution. One factor contained 10 items related to Faculty and Administrative 
management, whereas the other factor was associated with 12 items related to Faculty 
Relationships. Item 39,"Tries out new ideas with the faculty," loaded on both factors. 


Table 26 


Component Matrix Coefficients for Chair Self-Ratings of Administrative Methods Following 


Varimax Rotation (N = 734). 


Factors 

CSA Item 1 2 

42. Lets faculty know expectations 690 = .330 
44. Coordinates faculty work 659 333 
49. Maintains standards of performance 658 317 
36. Makes sound suggestions 655 9.226 
33. Allocates responsibilities 611 9.254 
47. Makes sure his/her part is understood 606 .419 
45. Explains the basis for decisions 083 0.247 
40. Sees that faculty are working 082 = 378 
37. Advocates dept. to admin 564 = =.086 
39. Tries out new ideas .460 .404 
51. Facilitates positive relationships .157 ~—-.690 
41. Looks out for fac. welfare 145.679 
38. Acts as though morale vital .267 668 
52. Encourages teamwork 295.657 
46. Lets members know good job 305 ~=.601 
43. Promotes inclusiveness and diversity 317.564 
34. Resolves conflicts 319 = .532 
53. Provides feedback 468 = .523 
35. Assists faculty goals 384 ~=.518 
48. Acts as though accomplishments vital 454  .499 
90. Puts suggestions into action 421.457 
Eigenvalues after rotation 4.978 4.767 
Percent of variance explained 23.66 22.70 


33 


Note. Number of factors were unrestricted. Boldface indicates highest factor loadings. Factor 1 = 


Faculty and Administrative Management; Factor 2 = Faculty Relationships 


Exploratory Factor Analysis of Faculty Ratings of Administrative Responsibilities 


of the chair's performance of administrative responsibilities. As shown in Table 27, the 
unrestricted principal component analysis revealed a unidimensional solution (eigenvalue= 


We also performed PCA (not forcing the number of factors extracted) on faculty ratings 


17.19), which we called Administrative Responsibilities and that explained 81% of the variance. 


Table 27 
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Component Matrix Coefficients for Faculty Ratings of the Chair's Performance of Administrative 


Responsibilities Following Varimax Rotation (N = 762). 


FPC Item Coefficients 
12. Fostering the development of each faculty member's special talents or interests .948 
15. Stimulating or rejuvenating faculty vitality/enthusiasm 943 
10. Guiding the development of a sound long-range plan to carry out departmental programs 932 
2. Fostering good teaching in the department 931 
5. Guiding the development of sound procedures for assessing faculty performance 931 
17. Establishing trust between himself/herself and members of the faculty 927 
4. Leading in establishing and monitoring progress on annual or biannual department goals 926 
8. Developing collegiality/cooperation among faculty members 919 
16. Facilitating curriculum development .918 
14. Clearly communicating expectations of the campus administration to the faculty 911 
20. Ensuring the assessment of student learning outcomes is meaningful and ongoing .907 
11. Promoting a positive image of the department within the campus community 905 
19. Rewarding faculty in accordance with their contributions to the department .903 
7. Communicating the department's needs to the dean and other appropriate administrators .902 
9. Stimulating research, scholarly activity, and/or creative endeavors in the department .899 
21. Actively supporting student recruitment and retention efforts .888 
13. Ensuring that new faculty and staff are acquainted with departmental procedures .884 
1. Attending to essential administrative tasks .879 
18. Promoting a positive image of the department to off-campus constituencies 875 
6. Facilitating successful recruitment and selection of promising faculty .864 
3. Assisting in securing funding from external sources .793 
Eigenvalue 17.19 
Percent of variance explained 81.43 


Note. Number of factors were unrestricted. 


Exploratory Factor Analysis of Faculty Ratings of Personal Characteristics 


We then entered faculty mean ratings of the 11 chair personal characteristics into a PCA. 
The component matrix coefficients and associated statistics presented in Table 28 showed 
support for a single dimension (eigenvalue = 9.09), Personal Characteristics, which explained 


82.59% of the variance. 


Component Matrix Coefficients for Faculty Ratings of Personal Characteristics (N = 762) 


Table 28 
FPC item Coefficient 
24. Practical judgment .967 
29. Consistency 944 
27. Fairness 943 
32. Clarity 942 
22. Problem solving ability 935 
25. Trustworthy 935 
26. Flexibility/adaptability .930 
23. Demonstrates caring .881 
31. Institution-centered .869 
30. Enterprising 824 
28. Organizational skills 811 
Eigenvalue 9.09 
Percent of variance explained 82.59 


Exploratory Factor Analysis of Faculty Ratings of Administrative Methods 


We also applied PCA to aggregated faculty ratings of the 21 administrative methods. 
The component matrix coefficients and associated statistics presented in Table 29 showed 
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support for a single dimension (eigenvalue = 17.99), Administrative Methods, which explained 


85.65% of the variance. 
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Table 29 
Component Matrix Coefficients for Faculty Ratings of 21 Administrative Methods (N = 762) 
FPC Item Coefficients 
47. Makes sure his/her part in the department is understood by all members .947 
52. Encourages teamwork among members of the faculty .946 
38. Demonstrates that high faculty morale is vital to him/her 944 
53. Provides feedback to faculty on their major activities 943 
36. Makes sound suggestions for developing/changing departmental priorities 942 
44, Makes sure the work of the faculty is coordinated .940 
49. Maintains definite standards of performance 939 
50. Puts faculty suggestions into action 939 
45. Explains the basis for his/her decisions .938 
35. Assists faculty in developing their own goals and priorities 937 
42. Lets faculty members know what is expected of them 931 
41. Looks out for the personal welfare of individual faculty members 924 
33. Allocates faculty responsibilities in an effective and equitable manner 923 
34. Reduces, resolves, and/or prevents conflict among departmental faculty members .923 
40. Sees to it that faculty members are working up to capacity 923 
48. Acts as though visible department accomplishments were vital to him/her .910 
39. Tries out new ideas with the faculty .903 
46. Lets faculty members know when they have done a good job .903 
43. Promotes inclusiveness and diversity among students and faculty .899 
51. Facilitates positive relationships between faculty and the clerical/technical staff 893 
37. Effectively advocates for departmental interests to higher authorities .883 
Eigenvalues 17,99 
Percent of variance explained 85.65 


Relationships between Faculty Ratings of Chair Performance and Summary Judgments 


An assumption underlying the IDEA system is that faculty ratings of the chair’s 


performance of responsibilities, personal characteristics, and administrative methods are related 
to faculty perceptions of the chair’s overall effectiveness. Faculty members respond to two 

questions on the FPC that assess their overall summary judgments: “I have confidence in the 
chair’s ability to provide future leadership to the department” (Item 54), and “Overall, this chair 


has provided excellent leadership” (Item 55). Pearson r correlations between faculty ratings of 
FPC items (in numerical sequence) and the two summary judgments are presented in Table 30. 


All correlations were significant (p < .001) and ranged from .73 to .94. The mean value of the 


correlations for Items 54 and 55 with all other FPC items were .87 and .88, respectively. 
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Table 30 

Pearson Correlations between Faculty Ratings of FPC Items and Overall Summary Judgments 

(N = 762) 

[Pearson r 

FPC Item Item 54 Item 55 
1. Attending to essential administrative tasks 285 .86 
2. Fostering good teaching in the department 89 .90 
3. Assisting in securing funding from external sources 73 74 
4. Leading in establishing and monitoring progress on annual or biannual 87 88 
5. Guiding the development of sound procedures for assessing faculty Rests} .89 
6. Facilitating successful recruitment and selection of promising faculty 82 82 
7. Communicating the department's needs to the dean and other appropriate 86 87 
8. Developing collegiality/cooperation among faculty members .90 91 
9. Stimulating research, scholarly activity, and/or creative endeavors in the 84 86 
10. Guiding the development of a sound long-range plan to carry out 189 .90 
11. Promoting a positive image of the department within the campus Kets} .88 
12. Fostering the development of each faculty member's special talents or OT 92 
13. Ensuring that new faculty and staff are acquainted with departmental 82 83 
14. Clearly communicating expectations of the campus administration to the |.86 87 
15. Stimulating or rejuvenating faculty vitality/enthusiasm .91 .92 
16. Facilitating curriculum development 87 87 
17. Establishing trust between himself/herself and members of the faculty .92 .93 
18. Promoting a positive image of the department to off-campus constituencies | 83 84 
19. Rewarding faculty in accordance with their contributions to the department |,85 86 
20. Ensuring the assessment of student learning outcomes is meaningful and {|,85 .86 
21. Actively supporting student recruitment and retention efforts 82 83 
22. Problem solving ability .91 91 
23. Demonstrates caring .83 .85 
24. Practical judgment .92 .94 
25. Trustworthy .89 .90 
26. Flexibility/adaptability .89 89 
27. Fairness .89 .90 
28. Organizational skills .76 77 
29. Consistency .88 .89 
30. Enterprising .81 .82 
31. Institution-centered .B1 .83 
32. Clarity .89 .90 
33. Allocates responsibilities .88 89 
34. Resolves conflicts 88 .09 
35. Assists faculty goals .89 89 
36. Makes sound suggestions .93 .93 
37. Advocates dept. to admin .86 87 
38. Acts as though morale vital .90 .91 
39. Tries out new ideas 185 Rsjs) 
40. Sees that fac. are working .86 .87 
41. Looks out for fac. welfare .88 .88 
42. Lets fac. know expectations .88 89 
43. Promotes inclusiveness and diversity .85 .86 
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44. Coordinates fac. work .90 .9O1 
45. Explains the basis for decisions .89 91 
46. Lets members know good job .03 .84 
47. Makes sure his/her part is understood .89 LOL 
48. Acts as though accomplishments vital .86 .O7 
49. Maintains standards of performance .90 .91 
50. Puts suggestions into action .90 .90 
51. Facilitates positive relationships .85 .86 
52. Encourages teamwork .90 .9O1 
53. Provides feedback .89 .90 


Note. Item 54 reads “I have confidence in the chair's ability to provide future leadership to the department”; Item 


55” is “Overall, this chair has provided excellent leadership.” 


Some items were more highly correlated with the two summary judgments than others. 
Among administrative responsibilities, establishing trust, stimulating faculty enthusiasm, 
fostering faculty talents, and developing collegiality were most highly correlated. In contrast, 
assisting in securing external funding had the lowest correlations. With respect to personal 


characteristics, problem solving ability and practical judgment were most influential. Concerning 


administrative methods, six were also highly correlated (>.90) with the summary judgments: 
makes sound suggestions, acts as though faculty morale is vital, coordinates faculty work, 
maintains standards of performance, puts suggestions into action, and encourages teamwork. 


Table 31 presents Pearson r correlation coefficients between mean scores on the three 
FPC scales (derived from the factor analyses) and the two overall summary judgments. Mean 
faculty ratings on the 21-item administrative responsibility scale, the 11-item personal 
characteristics scale, and the 21-item administrative methods scale were all highly correlated 
with the two summary judgment items (correlations ranged from .95 to .96). 


Table 31 


Pearson Correlations between Faculty Ratings of Responsibilities, Personal Characteristics, 


Administrative Methods, and Overall Summary Judgments (N = 762) 


Pearson r 
[Administrative Responsibilities Item 54 ([tem55 
21-item scale .95 .96 
Personal Characteristics 
11-item scale .95 .96 
[Administrative Methods 
21-item scale .95 .96 


Note. Item 54 reads “I have confidence in the chair’s ability to provide future leadership to the department”; Item 55 
reads “Overall, this chair has provided excellent leadership.” 


In sum, the magnitudes of these correlations clearly show that faculty ratings of the 
chair's performance are related to their overall impressions of the department chair. As faculty 


ratings of the chair’s performance on administrative responsibilities, personal characteristics, and 
administrative methods increase, faculty overall impressions of the chair’s performance also tend 
to increase. 
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Correlations with Other Variables 


Relationships between Faculty Ratings of Administrative Methods/Personal 
Characteristics and Administrative Responsibilities 


Another source of evidence regarding internal structure is found in the relationships 
between faculty ratings of administrative methods/personal characteristics and administrative 
responsibilities. An assumption underlying the FSC is that relationships between faculty ratings 
of administrative methods/personal characteristics and administrative responsibilities are 
distinctive for each responsibility. The Chair Report, therefore, includes a diagnostic section that 
lists high, moderate, and low priority responsibilities with their respective personal 
characteristics and administrative methods associated with high performance. Those groupings 
are based on the analyses described in the following paragraphs. 


We first correlated faculty ratings of the chair’s personal characteristics and 
administrative methods with their ratings of the 21 responsibilities. For each responsibility, we 
controlled for the chair’s priority rating by selecting for the analysis only those chairs who gave a 
rating of “moderate priority” or “high priority.” As shown in Table 32, these correlations were 
positive and moderate-to-high in magnitude (Pearson rs ranged from .63 to .95), which supports 
the internal structure of the CSA and FPC. 


Table 32 
Correlations between Faculty Ratings of Administrative Responsibilities and Personal 
Characteristics/Methods When the Chair Rated Responsibility as a Moderate or High Priority 


FPC Administrative Responsibility 


Item 1 2 8B 4 6 6 7 8 10 1 (12 93 14 115 116 7 18 119 20 21 
In 746 |735 [542 {728 1685 [716 |752 {738 |712 (729 {730 {726 |721 |733 |704 |702 {748 |705 (698 693 
22 84 185 1.75 185 185 180 184 184 181 187 183 185 180 184 185 185 185 180 181 183 
23 72.179 |63 170 1.77 172 1.73 1.88 172 174 1.79 (86 1.74 175 185 1.75 190 176 178 171 
24 85 185 |.71 1.83 186 179 183 188 180 185 185 188 181 185 187 184 192 181 183 181 
25 79 1.82 |66 177 |.83 177 |.78 191 175 179 |80 187 1.79 180 186 179 195 176 181 176 
26 76 |.82 |68 176 1.82 176 |.79 189 178 180 |83 189 |.79 180 188 181 192 179 181 176 
27 78 1.83 |67 177 |.84 178 |.79 |91 177 180 |81 189 180 181 188 181 194 178 185 177 
28 87 173 |.59 1.79 176 164 1.70 168 165 176 168 169 174 177 168 1.71 168 164 165 175 
29 86 184 1.66 182 185 175 1.78 186 175 182 180 185 182 184 184 181 189 175 181 181 
30 71 1.77 |82 (82 1.76 176 |.80 1.75 182 183 179 179 1.71 173 181 179 172 181 175 177 
St 78 1.79 |68 (80 1.79 174 1.77 1.78 173 180 |82 179 1.73 1.78 178 177 177 1.81 173 180 
32 85 183 1.65 [82 184 174 180 186 176 184 181 185 183 187 185 183 189 176 179 181 
33 83 184 |.71 [82 186 179 180 188 181 183 180 188 182 185 188 184 189 178 184 180 
34 78 |.84 |68 179 |.84 178 |.78 |92 179 181 |81 189 179 [181 188 183 191 1.77 182 178 
35 78 |.87 1.75 184 [87 179 |83 1.87 186 185 181 193 [81 183 189 185 188 179 185 181 
36 83 186 |.76 189 187 181 185 187 184 192 1.84 [88 183 185 189 188 188 181 182 184 
Bz 76 |.80 1.78 (83 |.81 178 |.93 1.79 180 185 184 183 1.76 181 183 180 180 182 179 177 
38 77 1.86 |.71 181 [85 179 |.81 1.93 180 183 185 (91 1.78 181 193 184 192 181 184 181 
39 74 1.83 |.74 (83 [81 178 |80 182 182 184 |81 186 1.77 180 187 186 180 179 180 179 
40 78 185 1.75 184 |86 179 |81 184 182 186 |80 188 180 181 187 185 182 177 183 184 
41 75 183 1.70 176 182 177 |.79 |90 179 179 |81 190 1.78 179 189 180 191 1.78 183 178 
42 84 186 |.69 185 188 177 1.81 185 180 184 180 187 185 187 185 185 186 176 182 183 
Wad 
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45 1.82 185 169 182 185 1.78 182 188 180 184 181 188 183 187 187 184 191 176 184 [80 
46.72 182 168 176 |.80 174 177 184 178 176 179 186 1.76 178 185 178 184 177 182 176 
47 1.84 187 1.71 185 |87 1.76 184 187 182 186 183 188 185 189 187 186 188 180 182 183 
48.77 183 1.76 183 182 178 183 182 (81 183 184 184 1.77 181 183 181 181 182 180 181 
49 |.84 188 1.75 188 |88 1.79 183 185 183 188 184 187 182 185 185 185 185 181 183 186 
50.79 184 1.72 182 185 180 183 187 181 184 182 190 182 183 187 184 189 179 183 181 
ol. 79 181 164 176 179 172 177 184 174 177 180 184 178 179 182 1.78 184 175 178 175 
p2..80 187 1.73 182 185 179 181 192 182 184 184 190 181 183 190 185 189 180 184 183 
53 83 187 1.73 186 188 179 183 185 183 185 182 189 183 186 187 184 187 180 184 182 


Note. n varied for each column, depending on number of chairs rating responsibility as Moderate Priority or High 


Priority. 


Administrative Responsibilities 


. Attending to essential administrative tasks 

. Fostering good teaching in the department 

. Assisting in securing funding from external sources 

. Leading in establishing and monitoring progress on goals 

. Guiding the procedures for assessing faculty performance 

. Facilitating recruitment and selection of promising faculty 
. Communicating the department's needs to the dean 

. Developing collegiality/cooperation among faculty 

. Stimulating research, scholarly activity 

10. Guiding the development of a sound long-range plan 


11. Promoting a positive image of the department within the 
campus community 


OOAN AUN BRWNY 


12. 
13. 
14. 
15. 
16. 
17. 
18. 
19. 
20. 


2 


pa 


Fostering development of faculty talents or interests 
Ensuring new faculty are acquainted with procedures 
Clearly communicating expectations of administration 
Stimulating or rejuvenating faculty vitality/enthusiasm 
Facilitating curriculum development 

Establishing trust between himself/herself and faculty 
Promoting a positive image of department off-campus 
Rewarding faculty in accordance with their contributions 
Ensuring the assessment of student learning outcomes 


. Actively supporting student recruitment and retention e 


Personal Characteristics 


22. Problem solving ability 
23. Demonstrates caring 
24, Practical judgment 

25. Trustworthy 


28. 
29. 
30. 
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pam 


Organizational skills 
Consistency 
Enterprising 


. Institution-centered 


26. Flexibility/adaptability 32. Clarity 

27, Fairness 

Administrative Methods 

33. Allocates responsibilities 44, Coordinates fac. work 

34. Resolves conflicts 45. Explains the basis for decisions 
35. Assists faculty goals 46. Lets members know good job 


36. Makes sound suggestions 

37. Advocates dept. to admin 

. Acts as though morale vital 

. Tries out new ideas 

. Sees that fac. are working 

. Looks out for fac. welfare 

. Lets fac. know expectations 

. Promotes inclusiveness and diversity 


47. 
48. 
49. 
50. 
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= 


Makes sure his/her part is understood 
Acts as though accomplishments vital 
Maintains standards of performance 
Puts suggestions into action 


. Facilitates positive relationships 
52. 
53. 


Encourages teamwork 
Provides feedback 


We then employed Bayesian Model Averaging (BMA) to investigate which chair 
personal characteristics and administrative methods were most important for explaining faculty 
ratings of each administrative responsibilities and the overall summary items. BMA is an 
ensemble technique that tests multiple models to obtain better predictive performance than what 
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could be obtained with a single model (Hoeting, Madigan, Raftery, & Volinsky, 1999). As stated 
previously, the analysis was limited to cases where chairs rated administrative responsibilities as 
a moderate or high priority. Separate analyses were, therefore, conducted on each administrative 
responsibility and the two summary items. The results of these analyses underlie the “Insights on 
Improvement” section of the Chair Report. 

Tables of estimated probabilities and regression parameters (weighted coefficients) are 
presented in Appendix A for each administrative responsibility and the two summary 
judgements. Table 33 summarizes the significant explanatory variables (indicated by item 
number) included in the “best” full models for each administrative responsibility and the 
summary measures. The chair personal characteristic “flexibility/adaptability in dealing with 
individuals/situations” (Item 26) was the only explanatory variable that was not included in any 
model. 

The personal characteristics and administrative methods significantly related to 
performance ratings of administrative responsibilities followed predictable patterns. For 
example, several administrative methods explained significant variance in faculty ratings of the 
chair’s performance in fostering good teaching in the department: "Demonstrates that high 
faculty morale is vital to him/her," "Makes sure the work of the faculty is coordinated," 
"Maintains definite standards of performance," and "Provides feedback to faculty on their major 
activities." In contrast, two different methods were important for the responsibility of 
"Communicating the department's needs to the dean and other appropriate administrators": 
"Effectively advocates for departmental interests to higher authorities," and "Makes sure his/her 
part in the department is understood by all members." Other examples of logical relationships 
may be found in Table 33. 


Table 33 
Chair Personal Characteristics and Administrative Methods Significantly Related to Chair Performance of Moderate or High Priority 
Administrative Responsibilities 


Administrative responsibility 


Personal characteristic or 
administrative method 


ONRDURWNE 


PPP rPrPrPrPrPrPrR wo 
WOAN AU BRWNRO* 


. Attending to essential administrative tasks 
. Fostering good teaching in the department 
. Assisting in securing funding from external sources 


Leading in establishing and monitoring progress on annual or biannual department goals 


. Guiding the development of sound procedures for assessing faculty performance 
. Facilitating successful recruitment and selection of promising faculty 


Communicating the department's needs to the dean and other appropriate administrators 


. Developing collegiality/cooperation among faculty members 


Stimulating research, scholarly activity, and/or creative endeavors in the department 
. Guiding the development of a sound plan to carry out departmental programs 

. Promoting a positive image of the department within the campus community 

. Fostering the development of each faculty member's special talents or interests 

. Ensuring that new faculty and staff are acquainted with departmental procedures 
. Clearly communicating expectations of the campus administration to the faculty 
. Stimulating or rejuvenating faculty vitality/enthusiasm 

. Facilitating curriculum development 

. Establishing trust between himself/herself and members of the faculty 

. Promoting a positive image of the department to off-campus constituencies 

. Rewarding faculty in accordance with their contributions to the department 


25, 28, 33, 44, 51 
38, 44, 49, 53 

50; 33,37 

28, 30, 31, 36, 49 
28, 35, 49 

25,30,37 

a7,47 

25, 34, 38, 52 

30, 35 

30, 36, 37, 44 

24, 31, 37, 38, 48 
33, 35, 41, 50 

28, 42, 47 

28, 31, 32, 33, 39, 45, 47 
30, 33, 38, 39 

22, 36, 39, 44 

23, 25, 32, 34, 36, 38 
30, 31, 37, 43 

27, 33, 35, 40, 46 


20. Ensuring the assessment of student learning outcomes is meaningful and ongoing 28, 31, 40, 44 

21. Actively supporting student recruitment and retention efforts 41 

54. Confidence in leadership 22, 25, 36, 37, 38, 52 
do. Excellent Leadership 22, 24, 25, 36, 37, 38 


Personal Characteristics 
22. Problem solving ability 28. Organizational skills 
23. Demonstrates caring 29. Consistency 
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24. Practical judgment 30. Enterprising 

25. Trustworthy 31. Institution-centered 

26. Flexibility/adaptability 32. Clarity 

27, Fairness 

Administrative Methods 

33. Allocates responsibilities 44, Coordinates fac. work 

34. Resolves conflicts 45. Explains the basis for decisions 

35. Assists faculty goals 46. Lets members know good job 

36. Makes sound suggestions 47. Makes sure his/her part is understood 
37. Advocates dept. to admin 48. Acts as though accomplishments vital 


38 
39 


. Acts as though morale vital 

. Tries out new ideas 

. Sees that fac. are working 

. Looks out for fac. welfare 

. Lets fac. know expectations 

. Promotes inclusiveness and diversity 


49. 
50. 


51 


Maintains standards of performance 
Puts suggestions into action 


. Facilitates positive relationships 
52. 
53. 


Encourages teamwork 
Provides feedback 
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Summary 


Several sources of validity evidence support the IDEA FCS. First, evidence for 
content validity can be found in a previous technical report (Benton et al., (2010). Second, 
principal components analysis of chair priority ratings revealed five dimensions underlying 
administrative responsibilities: Program Leadership, Personnel Management, Developing 
Positive Climate, Building Image/Reputation, and Administrative Support. Third, chair self- 
ratings of performance of administrative responsibilities, personal characteristics, and 
administrative methods were multidimensional. Fourth, faculty ratings of the chair’s 
performance of administrative responsibilities, personal characteristics, and administrative 
methods were all represented by a single dimension. Fifth, faculty performance ratings for all 
FPC items were highly correlated with two summary judgments. Sixth, Bayesian Model 
Averaging analysis, controlling for the chair’s priority ratings, revealed that the most 
important personal characteristics and administrative methods associated with faculty ratings 
of the chair’s performance were distinctive for each responsibility. 


Although validity evidence has been presented with respect to the IDEA FSC, the 
ultimate test of an instrument’s validity is in its use. To make appropriate use of the system 
more achievable, the user is encouraged to read about best practices in the Feedback System 
for Chairs. 


RELIABILITY AND STANDARD ERRORS OF MEASUREMENT 


Reliability refers to consistency in scores across repeated administrations of an 
assessment instrument (AERA, APA, NCME, 2014). In this section, evidence for reliability of 
the FPC is shown in how consistently faculty members within the same department rate the 
chair’s performance, and in the degree of stability in ratings of the same chairs on two different 
occasions. Evidence is also presented for the internal consistency of FPC subscales. Standard 
errors of measurement indicate that all scores contain some amount of unexplained variance, or 
error. Users should, therefore, recognize the imperfect nature of any measure when interpreting 
scores provided in the Chair Report. 


Within-Group Interrater Reliability and Standard Errors of Measurement 


Feedback is more helpful in decision making if faculty ratings within a department are 
fairly consistent. If they are inconsistent, the chair will be uncertain about what actions to take. 
To examine the consistency of faculty ratings within the same department, we applied the 
procedures described in James, Demaree, and Wolf (1984, p. 87) for computing the within-group 
interrater reliability coefficient. We began by selecting chairs for whom the number of raters 
equaled 12 or more. There were 397 chairs who met this criterion. Following that we estimated 
coefficients for chairs with 8, 16, and 20 respondents, using the Spearman-Brown prophecy 
formula. For each chair with more than 12 raters, 12 were randomly selected. We then applied 
the following single-item interrater agreement formula to compute the reliability coefficients for 
faculty ratings of chair administrative responsibilities, personal characteristics, and 
administrative methods at the department level: 


45 


rwcW) = 1— (Sxj7/oEU’) 


where rwac) is the within-group interrater reliability for a group of K judges on a single item Xj, 
and Sy is the observed variance of Xj. czu’ is the variance of X; that would be expected if all 
judgments solely resulted from random measurement error. Values range from .00 to 1.00, with 
higher coefficients indicative of greater internal consistency. 


The Spearman-Brown prophecy formula (show below) was then used to estimate the reliability 
of the ratings for administrators with 8, 16, and 20 raters: 


Vx = nr 


1+(n-1)r 


Next, we computed the standard error of measurement on each item, applying the following 
formula: 
SEM = s.d.[V(1 —rxx)] 


where rxx denotes the single-item reliability. SEM provides an estimate of the amount of error, at 
the 68th percent confidence interval, that would likely be associated with the obtained mean 
score on an individual item.* 


Table 34 presents the reliability coefficients and SEM for each item on the FPC. The 
reliability coefficients when the number of raters = 12 were at or above .70, and all SEM were 
below 0.4 with the exception of one item: Establishing trust between himself/herself and 
members of the faculty (SEM = 0.44). Applying the Spearman-Brown formula, the estimated 
reliability coefficients for 8 raters exceeded .50 except for two items: Assisting in securing 
funding from external sources and Establishing trust between himself/herself and members of the 
faculty. All SEM were at or below .54 except for two items: Assisting in securing funding from 
external source, and Establishing trust between himself/herself and members of the faculty. 
When there were at least 16 faculty raters, the estimated reliability coefficient was .77 or higher 
and SEM .37 or less except for two items: Assisting in securing funding from external source, 
and Establishing trust between himself/herself and members of the faculty. Finally, the estimated 
reliability coefficients for 20 raters was .80 or higher and SEM .33 or less except for two items: 
Assisting in securing funding from external source, and Establishing trust between 
himself/herself and members of the faculty. 


? To obtain the 90th percent confidence interval, multiply SEM coefficients in Table 34 by 1.645. 
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Table 34 
Within-group Interrater Reliability Coefficients for 12 Respondents (rwg16), Estimates of Reliabilities 
for 8, 16, and 20 Respondents by Spearman-Brown Prophecy Formula (r8, r16, and r20), and 


Standard Errors of Measurement for Items on the Faculty Perceptions of Chair Instrument (N = 397) 


Item SD re SEMs riz SEMi2_ rie SEMie_ r20 SEMz20 
1. Attending to essential administrative tasks 23 0.57 67 0.35 84 0.23 86 0.23 88 0.21 
2. Fostering good teaching in the department 408 0.57 64 O38 84 0.23 84 0.25 87 0.23 
3. Assisting in securing funding from external sources 3.70 0.73 39 0.68 74 0.37 69 0.48 .74 0.45 
4. Leading in establishing and monitoring progress on department goals 401 0.63 60 0.42 80 0.28 82 0.28 85 0.26 
5. Guiding the development of sound procedures for assessing faculty 3.88 0.66 56 0.47 78 0.31 80 0.32 83 0.29 
6. Facilitating successful recruitment and selection of promising faculty 4.03 0.65 56 0.47 79 0.30 80 0.32 83 0.29 
7. Communicating the department's needs to the dean 4.17 0.58 61 0.41 83 0.24 83 0.27 86 0.25 
8. Developing collegiality/cooperation among faculty members 392 0.74 52 0.52 73 0.38 77 0.36 81 0.33 
9. Stimulating research, scholarly activity, and/or creative endeavors 3.89 0.64 60 0.43 79 0.29 82 0.28 85 0.26 
10. Guiding the development of a plan to carry out programs 3.89 0.68 56 0.47 77 0.33 80 0.31 83 0.29 
11. Promoting a positive image within the campus community 4.22 0.61 59 0.43 81 0.27 82 0.29 85 0.27 


12. Fostering the development of each faculty member's special talents 3.89 0.66 57 0.45 78 0.31 B81 0.31 84 0.28 
13. Ensuring that new faculty and staff are acquainted with procedures 4.01 0.61 54 0.50 81 0.26 79 0.34 82 0.31 
14. Clearly communicating expectations of the campus administration 4.09 0.58 65 0.37 83 0.24 85 0.24 87 0.22 


15. Stimulating or rejuvenating faculty vitality/enthusiasm 3.67 0.74 OL 0.53 72 0.39 77 ~=0.36 81 0.33 
16. Facilitating curriculum development 3.95 0.61 .60 0.42 81 0.27 82 0.28 85 0.26 
17. Establishing trust between himself/herself and members of the faculty 3.93 0,79 A5 0.60 69 0.44 73 0.42 ST 0.39 
18. Promoting a positive image to off-campus constituencies 4.25 0.59 51 0.53 82 0.25 77 0.37 81 ~~ 0.33 
19. Rewarding faculty in accordance with their contributions 2.76 0.66 56 0.47 .78 0.31 80 0.32 83 0.29 
20. Ensuring the assessment of student learning outcomes is meaningful 400 0.59 59 0.44 83 0.24 81 0.30 85 0.27 
21. Actively supporting student recruitment and retention efforts 4.14 0.58 54 0.49 83 0.24 79 0.34 «2.82 ~ = 0.31 
22. Problem solving ability 4.14 0.60 64 0.38 82 0.26 84 0.25 87 = 0.23 
23. Demonstrates caring 4.17 0.66 7 ~=60..46 78 0.31 80 = 0.31 84 0.28 
24, Practical judgment 4.16 0.61 .62 0.40 81 0.27 83 0.26 £86 0.24 
25. Trustworthy 4.17 0.68 06 ~=60..47 77 ~~ (0.32 820 = 0.32 83 0.29 
26. Flexibility/adaptability 4.09 0.66 08 0.44 78 ~=0.30 .81 0.30 84 0.27 


27. Fairness 4.11 064 61 O42 80 0.29 82 0.28 85 0.25 


28. 
29. 
30. 
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pay 


Organizational skills 
Consistency 
Enterprising 


. Institution-centered 
32. 
33. 
34. 
35. 
36. 
37. 
38. 
39. 
40. 


Clarity 

Allocates responsibilities 
Resolves conflicts 

Assists faculty goals 
Makes sound suggestions 
Advocates dept. to admin 
Acts as though morale vital 
Tries out new ideas 

Sees that fac. are working 


. Looks out for fac. welfare 
42. 
43. 
44. 


Lets fac. know expectations 
Promotes inclusiveness and diversity 
Coordinates fac. work 


. Explains the basis for decisions 
46. 
47. 
48. 
49. 
50. 


Lets members know good job 

Makes sure his/her part is understood 
Acts as though accomplishments vital 
Maintains standards of performance 
Puts suggestions into action 


. Facilitates positive relationships 
52. 
53. 
54. 
55. 


Encourages teamwork 
Provides feedback 
Confidence in leadership 
Excellent leadership 


4.10 
4.10 
4.04 
4.26 
4.02 
3.94 
3.85 
oie Pe 
4.01 
4.19 
3.91 
4.00 
3.89 
4.01 
4.07 
4.21 
ee Hl 
4.05 
4.14 
4.04 
4.21 
4.08 
4.00 
4.18 
4.05 
4.05 
4.12 
4.07 


0.64 
0.63 
0.62 
0.50 
0.64 
0.61 
0.73 
0.63 
0.64 
0.59 
0.73 
0.62 
0.58 
0.65 
0.57 
0.56 
0.62 
0.65 
0.58 
0.61 
0.56 
0.60 
0.62 
0.61 
0.63 
0.59 
0.69 
0.69 


61 
61 
og 
.66 
.60 
61 
ol 
wg 
61 
mae) 
00) 
63 
62 
03 
65 
62 
61 
62 
61 
62 
64 
64 
.60 
08 
ae 
63 
04 
“0 


0.41 
0.42 
0.43 
0.36 
0.42 
0.41 
0.53 
0.43 
0.41 
0.43 
0.54 
0.39 
0.40 
0.51 
0.36 
0.40 
0.41 
0.40 
0.41 
0.40 
0.38 
0.38 
0.43 
0.45 
0.43 
0.39 
0.49 
0.48 


Ag 
.80 
81 
87 
be) 
81 
73 
.80 
.80 
82 
fo 
81 
83 
mips 
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Internal Consistency of Factor Scale Scores and Overall Scales 


Another estimate of internal consistency is Cronbach’s coefficient alpha (a) (Cronbach, 
1951) or the intraclass coefficient, which is reflective of the average inter-correlation among 
items. Cronbach's « is a ratio of (a) the number of items in the subscale squared and then 
multiplied by the average covariance between items, and (b) the sum of all subscale item 
variances and covariances. Again, values range from .00 to 1.00. Although neither factor scores 
nor scale scores are provided in the Chair Report, we computed the intraclass coefficients to 
offer additional evidence of the internal consistency of the dimensions of the CSA and FPC. 


Internal Consistency of CSA Factors Scales 


First, we computed Cronbach o coefficients for the factor scales revealed in the principle 
components analysis (PCA) of chair priority ratings of administrative responsibilities (see Table 
35). All but the “Administrative Support” scale had intraclass coefficients greater than or equal 
to .69. 


Table 35 

Cronbach a Coefficients for the Five Factors of Chair Priority Ratings of Administrative 

Responsibilities 
Factor Cronbach a 
Program Leadership 74 
Personnel Management .70 
Developing Positive Climate ml 
Building Image/Reputation .69 
Administrative Support 02 


Next, we calculated intraclass coefficients for the factor scales found in the PCA chair 
self-ratings of administrative responsibilities (see Table 36). Three factor scales showed 
acceptable internal consistency: Cronbach’s a ranged from .71 to .82, while two scales 
“Developing Positive Climate” and “Administrative Support” showed marginal internal 
consistency (Cronbach’s o = .66). 


Table 36 
Cronbach a Coefficients for the Five Factor Scales of Chair Self-Ratings of Administrative 
Responsibilities 

Factor Cronbach o 


Program Leadership 82 
Personnel Management 81 
Developing Positive Climate .66 
Building Image/Reputation 71 
Administrative Support 66 


We also tested the internal consistency of the two factor scales revealed in the PCAs of 
the chair self-ratings of personal characteristics (see Table 37) and administrative methods (see 
Table 38). The “Faculty Relationships” scale underlying personal characteristics showed 
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marginal internal consistency, whereas the “Faculty and Administrative Management” scale 
showed acceptable internal consistency. Both administrative methods scales showed high 
internal consistency. 


Table 37 
Cronbach a Coefficients for the Two Factor Scales of Chair Self-Ratings of Personal 
Characteristics 


Factor Cronbach o 

Faculty and Administrative Management he) 

Faculty Relationships .68 
Table 38 


Cronbach a Coefficients for the Two Factor Scales of Chair Self-Ratings of Administrative 
Methods 


Factor Cronbach a 
Faculty and Administrative Management 87 
Faculty Relationships 88 


Internal Consistency of FPC Scales 


We tested the internal consistency of the single dimension scales revealed in the PCAs of 
faculty ratings of chair performance of administrative responsibilities, personal characteristics, 
and administrative methods. Cronbach o coefficient for the single dimension faculty ratings of 
performance on the 21 administrative responsibilities was .99, which indicates very high internal 
consistency. Internal consistency was also very high for both the 11-item personal characteristics 
scale (Cronbach’s o = .98) and the 21-item administrative methods scale (Cronbach’s a = .99). 
The very high internal consistency indicates ratings on all items within each of the respective 
scales are closely related. 


Estimates of Item Stability 


Stability refers to the consistency of a measure across time, which provides additional 
evidence of reliability. We examined the stability of the faculty ratings for 234 chairs who 
participated in the IDEA FSC on at least two different occasions. If a given chair participated 
more than twice, the participation closest in time to that of the database preparation was used. 
The time between ratings varied. Table 39 shows the Pearson r “test-retest” correlations for all 
items on the FPC. Coefficients ranged from .57 to .75, with a median of .61, which indicated that 
there was moderate-to-strong stability between performance ratings of the same department 
chairs. Extremely high test-retest correlations are not necessarily to be expected, because some 
chairs may make more improvements than others, which would reduce the magnitude of the 
correlations. 


Table 39 
Pearson r Correlations between Faculty Performance Ratings of the Same Chair on Two 
Different Occasions (N = 234) 


FPC Item Pearson r 
1. Attending to essential administrative tasks .64 
2. Fostering good teaching in the department .66 
3. Assisting in securing funding from external sources 04 
4. Leading in establishing and monitoring progress on department goals 63 
5. Guiding the development of sound procedures for assessing faculty 65 
6. Facilitating successful recruitment and selection of promising faculty .06 
7. Communicating the department's needs to the dean 61 
8. Developing collegiality/cooperation among faculty members .68 
9. Stimulating research, scholarly activity, and/or creative endeavors .66 
10. Guiding the development of a sound plan to carry out departmental programs 62 
11. Promoting a positive image of the department within the campus community 63 
12. Fostering the development of each faculty member's special talents .67 
13. Ensuring that new faculty and staff are acquainted with procedures 57 
14. Clearly communicating expectations of the campus administration .60 
15. Stimulating or rejuvenating faculty vitality/enthusiasm .70 
16. Facilitating curriculum development 62 
17. Establishing trust between himself/herself and members of the faculty .67 
18. Promoting a positive image of the department to off-campus constituencies ay 
19. Rewarding faculty in accordance with their contributions to the department Roy, 
20. Ensuring the assessment of student learning outcomes is meaningful 63 
21. Actively supporting student recruitment and retention efforts 64 
22. Problem solving ability 62 
23. Demonstrates caring .67 
24. Practical judgment 65 
25. Trustworthy 63 
26. Flexibility/adaptability 62 
27. Fairness 65 
28. Organizational skills 75 
29. Consistency .68 
30. Enterprising by 
31. Institution-centered .06 
32. Clarity .67 
33. Allocates responsibilities .68 
34. Resolves conflicts agi 
35. Assists faculty goals .68 
36. Makes sound suggestions 65 
37. Advocates dept. to admin 61 
38. Acts as though morale vital .70 
39. Tries out new ideas 62 
40. Sees that fac. are working 63 
41. Looks out for fac. welfare 65 


42. Lets fac. know expectations 62 


o1 


43. Promotes inclusiveness and diversity 08 
44. Coordinates fac. work 61 
45. Explains the basis for decisions .60 
46. Lets members know good job 61 
47. Makes sure his/her part is understood 61 
48. Acts as though accomplishments vital 56 
49. Maintains standards of performance .67 
50. Puts suggestions into action .61 
51. Facilitates positive relationships .60 
52. Encourages teamwork .64 
53. Provides feedback 207 
54. Confidence in leadership .64 
55. Excellent leadership .68 
Summary 


We presented three sources of evidence for the reliability of the FSC instruments. First, 
within-group interrater reliability was moderate to high at the department level for faculty ratings 
of the same chair on the same occasion. Similarly, standard error of measurement estimates 
supported the dependability of individual items. Second, intraclass reliability coefficients 
demonstrated acceptable internal consistency for the priority ratings of the five subscales of the 
CSA. Internal consistency was also acceptable for chair self-ratings of the 21-item administrative 
responsibilities, 11-item personal characteristics, and 21-item administrative methods subscales. 
Third, high internal consistency was shown for faculty ratings of the FPC scales. Finally, test- 
retest correlation coefficients revealed stability in faculty ratings of the same chair across time. 


CRITERION-REFERENCE INTERPRETATIONS 


Criterion-referenced interpretations are based on an individual’s standing with respect 
to some standard. The IDEA Chair Report supplies criterion-referenced interpretations for the 
administrative responsibilities, personal characteristics and administrative methods. In 
addition, gap scores reflect the difference between chair self-ratings and faculty perceptions of 
the chair’s performance. In the report, chairs are deemed to have achieved “positive” ratings on 
an administrative responsibility when at least 70% of the faculty rated the trait as either a "4" 
(Good) or “5" (Outstanding). For personal characteristic and administrative methods, 
"positive" ratings are also indicated when at least 70% of faculty have perceived the trait as "4" 
(More a strength than a weakness) or a "5" (Definite strength). The criterion of 70% is not 
arbitrary, but is instead informed by normative data. To paraphrase Angoff (1974), if you 
scratch a criterion, you will very likely find a norm. The "norms" for faculty ratings of 
performance were first derived from an analysis conducted by Benton et al. (2010) and were 
re-confirmed using the current sample. The percent of faculty rating the chair as “4” or “5” on 
each of the 21 responsibilities is shown in Table 40; the percentages for personal 
characteristics, administrative methods, and summary judgments are shown in Table 41. As 
shown at the bottom of the tables, the median percent of faculty responding “4” or “5S” across 
all administrative responsibility items was 70.5%. Similarly, the median percent of faculty 
responding “4” or “5” across all personal characteristic and administrative methods items was 
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73.7%. So, the standard of 70% remains a reasonable expectation. 


Gap scores between chair self-ratings and mean faculty ratings of performance 
provide another opportunity for criterion-referenced interpretations. Qualitative categories 
are used to position the chair relative to the faculty on each characteristic, using the 
following scale: 


“Higher” = The chair’s self-rating is at least one-half point (0.5) higher than the faculty 
average. 

“Similar” = The chair’s self-rating is within one-half-point (0.5) of the faculty average. 
“Lower” = The chair’s self-rating is at least one-half point (0.5) lower than the faculty average. 


The one-half point criterion is based on the logic that any gap less than 0.5 on a five-point scale 
is not a meaningful difference. A gap of 0.5 or greater typically would mean the majority of 
faculty are giving the chair a rating different from his or her self-rating. Table 42 shows the 
means and standard deviations for all administrative responsibilities, personal characteristics, and 
administrative methods gap scores. All gap scores were within the + 0.5 difference with one 
exception: On average, chairs tended to rate themselves at least 0.5 higher than did faculty on 
"Assisting in securing funding from external sources." All other gap scores were within the + 0.5 
difference. 


Table 40 
Percent of Faculty Rating Chair “Good” or “Outstanding” on Administrative Responsibilities 
(Calculated Using Individual Faculty as the Unit of Analysis) 


Faculty level data: 
Raw % “Good” and 


Item “Outstanding” 
1. Attending to essential administrative tasks 81.5 
2. Fostering good teaching in the department 75.3 
3. Assisting in securing funding from external sources 59.2 
4. Leading in establishing and monitoring progress on department goals 73.1 
5. Guiding the development of sound procedures for assessing faculty 68.1 
6. Facilitating successful recruitment and selection of promising faculty 72.1 
7. Communicating the department's needs to the dean 75.1 
8. Developing collegiality/cooperation among faculty members 70.5 
9. Stimulating research, scholarly activity, and/or creative endeavors 69.4 
10. Guiding the development of a sound plan to carry out departmental programs 68.5 
11. Promoting a positive image of the department within the campus community 79.2 
12. Fostering the development of each faculty member's special talents 68.7 
13. Ensuring that new faculty and staff are acquainted with procedures 68.7 
14. Clearly communicating expectations of the campus administration 75.6 
15. Stimulating or rejuvenating faculty vitality/enthusiasm 62.1 
16. Facilitating curriculum development 69.0 
17. Establishing trust between himself/herself and members of the faculty 71.0 
18. Promoting a positive image of the department to off-campus constituencies 76.2 
19. Rewarding faculty in accordance with their contributions to the department 62.2 
20. Ensuring the assessment of student learning outcomes is meaningful 69.9 
21. Actively supporting student recruitment and retention efforts 73.4 
Mean % 70.90 
Median % 70.50 
SD 9.48 
Note. Percentages are calculated without number of missing values in the denominator (equivalent to 


“Valid Percent” in SPSS terminology). 


Table 41 
Percent of Faculty Rating Chair Positively on Personal Characteristics, Administrative 
Methods, and Summary Judgments (Calculated Using Individual Faculty as the Unit of 
Analysis) 


Faculty level data: Raw % 


Item “Good” and “Outstanding” 
22. Problem solving ability 76.9 
23. Demonstrates caring 77.6 
24. Practical judgment 77.4 
25. Trustworthy Tho 
26. Flexibility/adaptability 74.5 
27. Fairness 75.6 
28. Organizational skills 76.7 
29. Consistency 75.8 
30. Enterprising 70.9 
31. Institution-centered 792 
32. Clarity Ta 
33. Allocates responsibilities 68.6 
34. Resolves conflicts 64.5 
35. Assists faculty goals 68.3 
36. Makes sound suggestions 72.0 
37. Advocates dept. to admin 75.1 
38. Acts as though morale vital 69.0 
39. Tries out new ideas 69.7 
AO. Sees that fac. are working 66.1 
41. Looks out for fac. welfare 70.9 
42. Lets fac. know expectations 73.9 
43. Promotes inclusiveness and diversity 77.1 
44. Coordinates fac. work 67.0 
45. Explains the basis for decisions 73.5 
46. Lets members know good job 75.9 
47. Makes sure his/her part is understood 72.6 
48. Acts as though accomplishments vital 76.2 
49. Maintains standards of performance 74,2 
50. Puts suggestions into action 70.4 
51. Facilitates positive relationships 76.2 
52. Encourages teamwork 72,9 
53. Provides feedback 72.2 
54. Confidence in chair’s leadership 75.2 
50. Chair has provided excellent leadership 73.5 
Mean % 73.30 
Median % 73.70 
SD fs) 


Note. Percentages are calculated without number of missing values in the denominator (equivalent to 
“Valid Percent” in SPSS terminology). 


Table 42 
Means and Standard Deviations for Gap Scores of Administrative Responsibilities, Personal 


Characteristics and Administrative Methods (N =762) 
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Item M SD 

1. Attending to essential administrative tasks 0.00 0.80 
2. Fostering good teaching in the department -0.23 0.90 
3. Assisting in securing funding from external sources -0.50 1.20 
4. Leading in establishing and monitoring progress on department goals -0.37 0.98 
5. Guiding the development of sound procedures for assessing faculty -0.25 1.05 
6. Facilitating successful recruitment and selection of faculty 0.04 1.06 
7. Communicating the department's needs to the dean 0.15 0.89 
8. Developing collegiality/cooperation among faculty members 0.06 0.93 
9. Stimulating research, scholarly activity, and/or creative endeavors -0.18 1.06 
10. Guiding the development of a sound plan to carry out programs -0.17 1.07 
11. Promoting a positive image of the department within the campus -0.04 0.95 
12. Fostering the development of each faculty member's special talents -0.07 0.97 
13. Ensuring that new faculty and staff are acquainted with procedures -0.16 1.02 
14. Clearly communicating expectations of the campus administration -0.06 0.92 
15. Stimulating or rejuvenating faculty vitality/enthusiasm -0.06 1.01 
16. Facilitating curriculum development -0.13 1.02 
17. Establishing trust between himself/herself and members of the faculty 0.21 0.88 
18. Promoting a positive image of the department to off-campus constituencies -0.18 0.98 
19. Rewarding faculty in accordance with their contributions to the department -0.12 1.11 
20. Ensuring the assessment of student learning outcomes is meaningful -0.42 1.03 
21. Actively supporting student recruitment and retention efforts -0.21 1.02 
22. Problem solving ability 0.31 0.84 
23. Demonstrates caring 0.11 0.90 
24. Practical judgment 0.19 0.85 
25. Trustworthy 0.46 0.82 
26. Flexibility/adaptability 0.13 0.94 
27. Faimess 0.40 0.86 
28. Organizational skills -0.15 1.01 
29. Consistency 0.11 0.94 
30. Enterprising -0.31 1,07 
31. Institution-centered -0.26 0.97 
32. Clarity -0.14 0.93 
33. Allocates responsibilities 0.07 0.88 
34. Resolves conflicts 0.05 0.94 
35. Assists faculty goals -0.05 0.98 
36. Makes sound suggestions 0.21 0.90 
37. Advocates dept. to admin 0.11 0.90 
38. Acts as though morale vital 0.18 0.96 
39. Tries out new ideas 0.00 0.92 
40. Sees that fac. are working -0.30 0.96 
41. Looks out for fac. welfare 0.15 0.94 
42. Lets fac. know expectations -0.06 0.90 
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43. Promotes inclusiveness and diversity -0.10 0.99 
44. Coordinates fac. work -0.13 0.95 
45. Explains the basis for decisions 0.10 0.92 
46. Lets members know good job 0.09 0.89 
47. Makes sure his/her part is understood -0.20 0.97 
48. Acts as though accomplishments vital -0.04 0.91 
49. Maintains standards of performance -0.04 0.98 
50. Puts suggestions into action 0.06 0.90 
51. Facilitates positive relationships -0.06 0.94 
52. Encourages teamwork 0.07 0.92 
53. Provides feedback -0.04 0.96 
SUMMARY 


The IDEA FSC is built upon a foundation of well-established items dating from the 
early 1970s to the current version. The purpose of this report was to convey findings from 
analyses conducted on data collected from 2013 to 2016. Evidence was presented to support the 
validity and reliability of the CSA and FPC instruments. 


Quantitative analyses were performed on aggregated data from 11,169 faculty members 
rating 762 different chairs from 54 institutions. The IDEA database represents a diverse group of 
users. Most departments in the database resided in an institution awarding either a master’s 
degree or the doctorate. Most institutions were public, although approximately a third were 
private institutions. All accrediting regions of the country were represented, but the North 
Central, Southern, and Middle States were most prominent. Chair years of service varied, but 
more than half had been in their current position for at least four years. 


Several kinds of validity evidence were presented. First, factor analysis provided 
evidence of multidimensionality in chair priority ratings of administrative responsibilities. In 
addition, chair self-ratings of performance on administrative responsibilities, personal 
characteristics, and administrative methods were multidimensional. In contrast, faculty ratings 
of the chair's performance were unidimensional. Second, Bayesian Model Averaging provided 
evidence of the relationships between faculty ratings of personal characteristics/administrative 
methods and responsibilities. Finally, evidence for the content validity of the items can be 
found in a previously published technical report (Benton et al., 2010). 


The reliability of items in the revised FSC instruments was supported by substantial 
evidence. Estimates of within group interrater reliability were moderate to high, depending on 
number of raters, and standard errors of measurement were low enough to support the 
dependability of interpretations based on individual item scores. Subscales derived from the 
factor analysis of the CSA generally showed high internal consistency, as did the overall 
scales of the FPC. Finally, faculty ratings of the chair's performance showed good stability 
across time for chairs rated on at least two occasions. 
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Within-group Interrater Reliability Coefficients on the Faculty Perceptions of Chair 
Instrument for Five or More Respondents and Eight or More Respondents 
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Within-group Interrater Reliability Coefficients on the Faculty Perceptions of Chair Instrument 


for Five or More Respondents and Eight or More Respondents 


n= 762 n= 615 

Item M SD r M SD r 

1. Attending to essential administrative tasks 3.88 0.66 .78 3.88 0.63 ~~ .80 
2. Fostering good teaching in the department 4.03 0.65 79 4.02 0.61 81 
3. Assisting in securing funding from external sources 4.23 0.57 84 4.24 0.54 85 
4. Leading in establishing and monitoring progress on department goals 4.08 0.57 84 4.08 0.55 85 
5. Guiding the development of sound procedures for assessing faculty 3.70 0.73 74 3.71 0.70 .76 
6. Facilitating successful recruitment and selection of promising faculty 4.01 0.63 80 4.02 0.60 82 
7. Communicating the department's needs to the dean 4.17 0.58 83 4.17 0.56 .84 
8. Developing collegiality/cooperation among faculty members 3.92 0.74 73 3.91 0.71 2/5 
9. Stimulating research, scholarly activity, and/or creative endeavors 3.89 0.64 79 3.90 0.61 81 
10. Guiding the development of a plan to carry out programs 3.89 0.68 WT 3.90 0.65 79 
11. Promoting a positive image within the campus community 4.22 0.61 81 4.21 060 #82 
12. Fostering the development of each faculty member's special talents 3.89 0.66 78 3.88 0.64 80 
13. Ensuring that new faculty and staff are acquainted with procedures 4.01 0.61 81 4.01 0.58 83 
14. Clearly communicating expectations of the campus administration 4.09 0.58 83 4.09 0.55 85 
15. Stimulating or rejuvenating faculty vitality/enthusiasm 3.67 0.74 72 3.66 0.72 74 
16. Facilitating curriculum development 3.95 061 81 3.94 0.59  .82 
17. Establishing trust between himself/herself and members of the faculty 3,93 0.79 .69 3.92 0.76 71 
18. Promoting a positive image to off-campus constituencies 4.25 0.59 82 4.24 0.58  .83 
19. Rewarding faculty in accordance with their contributions 3.76 0.66 .78 3.76 0.62 81 
20. Ensuring the assessment of student learning outcomes is meaningful 400 0.59 83 3.99 0.56 .84 
21. Actively supporting student recruitment and retention efforts 4.14 0.58 83 4.13 0.56 .84 
22. Problem solving ability 414 0.60 .82 414 0.58  .83 
23. Demonstrates caring 417 0.66 .78 417 0.65 ~ .79 
24. Practical judgment 416 0.61 81 415 0.60 82 
25. Trustworthy 417 0.68 .77 417 0.65  .79 
26. Flexibility/adaptability 4.09 0.66 .78 4.08 0.64 ~~ .80 
27. Fairness 411 0.64 £80 4.10 0.62 81 
28. Organizational skills 410 0.64 .79 412 0.62 81 
29. Consistency 410 0.63 .£80 4.09 0.60  .82 
30. Enterprising 404 0.62 81 403 0.58 83 
31. Institution-centered 4.26 0.50 87 425 0.48 88 
32. Clarity 402 0.64 .79 402 0.62 81 
33. Allocates responsibilities 3.94 0.61 81 3.93 0.58  .83 
34. Resolves conflicts 3.09. 0.73. 7/3 ~3:63: 0.71. 476 
35. Assists faculty goals 3.92 0.63 80 3.92 0.60  .82 
36. Makes sound suggestions 401 0.64 80 400 0.62 81 
37. Advocates dept. to admin 4.19 0.59 82 418 0.58  .83 
38. Acts as though morale vital 3.91 0.73 73 3.91 0.70 .75 
39. Tries out new ideas 400 0.62 81 3.99 0.59  .82 
40. Sees that fac. are working 3.89 0.58 83 3.87 0.56  .84 
41. Looks out for fac. welfare 4.01 065 .79 400 0.64 .80 
42. Lets fac. know expectations 4.07 0.57 4.84 406 0.54  .85 


43. 
44. 
45. 
46. 
47. 
48. 
49. 
50. 
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ay 


Promotes inclusiveness and diversity 
Coordinates fac. work 

Explains the basis for decisions 

Lets members know good job 

Makes sure his/her part is understood 
Acts as though accomplishments vital 
Maintains standards of performance 
Puts suggestions into action 


. Facilitates positive relationships 
52. 
53. 
54. 
55. 


Encourages teamwork 
Provides feedback 
Confidence in leadership 
Excellent leadership 


4.21 
3.91 
4.05 
4.14 
4.04 
4.21 
4.08 
4.00 
4.18 
4.05 
4.05 
4.12 
4.07 


0.56 
0.62 
0.65 
0.58 
0.61 
0.56 
0.60 
0.62 
0.61 
0.63 
0.59 
0.69 
0.69 


84 
81 
Wie) 
83 
81 
85 
82 
81 
81 
.80 
82 
.76 
.76 


4.21 
3.90 
4.03 
4.12 
4.02 
4.20 
4.08 
3.98 
4.17 
4.05 
4.04 
4.11 
4.06 
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0.54 
0.59 
0.62 
0.56 
0.59 
0.53 
0.56 
0.60 
0.61 
0.60 
0.57 
0.67 
0.67 


.86 
83 
81 
85 
83 
.86 
84 
82 
82 
82 
84 
ty: 
Nes 
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Bayesian Model Averaging Results Tables 


61 


62 


1. Attends to admin. details (n = 744) 

TERM Posterior effect probability Posterior Mean Posterior SD t 
Chair22 

Chair23 

Chair24 

Chair25 0.925 0.167 0.067 2.493 
Chair26 

Chair27 

Chair28 1.000 0.462 0.029 15.931 
Chair29 

Chair30 

Chair31 

Chair32 

Chair33 1.000 0.214 0.039 5.487 
Chair34 

Chair35 

Chair36 

Chair37 

Chair38 

Chair39 

Chair40 

Chair41 

Chair42 

Chair43 

Chair44 1.000 0.192 0.042 4.571 
Chair45 

Chair46 

Chair47 

Chair48 

Chair49 

Chair50 

Chair51 1.000 0.133 0.030 4.433 
Chair52 

Chair53 


2. Fosters good teaching (n = 733) 


TERM Posterior effect probability 


Chair22 
Chair23 
Chair24 
Chair25 
Chair26 
Chair27 
Chair28 
Chair29 
Chair30 
Chair31 
Chair32 
Chair33 
Chair34 
Chair35 
Chair36 
Chair37 
Chair38 
Chair39 
Chair40 
Chair41 
Chair42 
Chair43 
Chair44 
Chair45 
Chair46 
Chair47 
Chair48 
Chair49 
Chair50 
Chair51 
Chair52 
Chair53 


1.000 


1.000 


1.000 


0.989 


Posterior Mean 


0.181 


0.207 


0.185 


0.149 


Posterior SD 


0.042 


0.042 


0.043 


0.043 


t 


4.310 


4.929 


4.302 


3.465 
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3. Obtains grants (n = 541) 

TERM Posterior effect probability Posterior Mean Posterior SD t 
Chair22 

Chair23 

Chair24 

Chair25 

Chair26 

Chair27 

Chair28 

Chair29 

Chair30 1.000 0.504 02053": <3:508 
Chair31 

Chair32 

Chair33 

Chair34 

Chair35 0.986 0.199 0.061). . 3.262 
Chair36 

Chair37 1.000 0.237 0.050 4.740 
Chair38 

Chair39 

Chair40 

Chair41 

Chair42 

Chair43 

Chair44 

Chair45 

Chair46 

Chair47 

Chair48 

Chair49 

Chair50 

Chair51 

Chair52 

Chair53 


4. Monitors progress (n = 725) 

TERM Posterior effect probability Posterior Mean Posterior SD t 
Chair22 

Chair23 

Chair24 

Chair25 

Chair26 

Chair27 

Chair28 1.000 0.206 0.032 6.437 
Chair29 

Chair30 0.983 0.123 0.037 3.324 
Chair31 0.999 0.103 0.028 3.679 
Chair32 

Chair33 

Chair34 

Chair35 

Chair36 1.000 0.358 0.047 7.617 
Chair37 

Chair38 

Chair39 

Chair40 

Chair41 

Chair42 

Chair43 

Chair44 

Chair45 

Chair46 

Chair47 

Chair48 

Chair49 0.908 0.124 0.057 92.175 
Chair50 

Chair51 

Chair52 

Chair53 
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5. Assesses faculty performance (n = 682) 

TERM Posterior effect probability Posterior Mean’ PosteriorSD  t 
Chair22 

Chair23 

Chair24 

Chair25 

Chair26 

Chair27 

Chair28 1.000 0.121 0.030 4.033 
Chair29 

Chair30 

Chair31 

Chair32 

Chair33 

Chair34 

Chair35 1.000 0.181 0.040 4.525 
Chair36 

Chair37 

Chair38 

Chair39 

Chair40 

Chair41 

Chair42 

Chair43 

Chair44 

Chair45 

Chair46 

Chair47 

Chair48 

Chair49 1.000 0.242 0.045 5.378 
Chair50 

Chair51 

Chair52 

Chair53 
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6. Recruits promising faculty (n = 713) 

TERM Posterior effect probability Posterior Mean PosteriorSD_ t 
Chair22 

Chair23 

Chair24 

Chair25 0.988 0.194 0.054 3.593 
Chair26 

Chair27 

Chair28 

Chair29 

Chair30 0.999 0.201 0.044 4.568 
Chair31 

Chair32 

Chair33 

Chair34 

Chair35 

Chair36 

Chair37 1.000 0.190 0.044 4.318 
Chair38 

Chair39 

Chair40 

Chair41 

Chair42 

Chair43 

Chair44 

Chair45 

Chair46 

Chair47 

Chair48 

Chair49 

Chair50 

Chair51 

Chair52 

Chair53 


7. Communicates dept. needs (n = 749) 
TERM Posterior effect probability Posterior Mean’ Posterior SD t 
Chair22 

Chair23 

Chair24 

Chair25 

Chair26 

Chair27 

Chair28 

Chair29 

Chair30 

Chair31 

Chair32 

Chair33 

Chair34 

Chair35 

Chair36 

Chair37 1.000 0.668 0.030 22.267 
Chair38 

Chair39 

Chair40 

Chair41 

Chair42 

Chair43 

Chair44 

Chair45 

Chair46 

Chair47 0.999 0.166 0.036 4.611 
Chair48 

Chair49 

Chair50 

Chair51 

Chair52 

Chair53 
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8. Develops collegiality (n = 735) 

TERM Posterior effect probability Posterior Mean Posterior SD t 
Chair22 

Chair23 

Chair24 

Chair25 1.000 0.189 0.031 6.097 
Chair26 

Chair27 

Chair28 

Chair29 

Chair30 

Chair31 

Chair32 

Chair33 

Chair34 1.000 0.315 0.027 11.667 
Chair35 

Chair36 

Chair37 

Chair38 1.000 0.274 O03" 7,211 
Chair39 

Chair40 

Chair41 

Chair42 

Chair43 

Chair44 

Chair45 

Chair46 

Chair47 

Chair48 

Chair49 

Chair50 

Chair51 

Chair52 1.000 0.173 0.030 5.767 
Chair53 


9. Stimulates research (n = 709) 
TERM Posterior effect probability Posterior Mean Posterior SD t 
Chair22 

Chair23 

Chair24 

Chair25 

Chair26 

Chair27 

Chair28 

Chair29 

Chair30 1.000 0.299 0.038 7.868 
Chair31 

Chair32 

Chair33 

Chair34 

Chair35 1.000 0.374 0.041 9,122 
Chair36 

Chair37 

Chair38 

Chair39 

Chair40 

Chair41 

Chair42 

Chair43 

Chair44 

Chair45 

Chair46 

Chair47 

Chair48 

Chair49 

Chair50 

Chair51 

Chair52 

Chair53 


10. Guides organizational plan (n = 726) 
TERM Posterior effect probability 


Chair22 
Chair23 
Chair24 
Chair25 
Chair26 
Chair27 
Chair28 
Chair29 
Chair30 
Chair31 
Chair32 
Chair33 
Chair34 
Chair35 
Chair36 
Chair37 
Chair38 
Chair39 
Chair40 
Chair41 
Chair42 
Chair43 
Chair44 
Chair45 
Chair46 
Chair47 
Chair48 
Chair49 
Chair50 
Chair51 
Chair52 
Chair53 


0,992 


1.000 
0.996 


0.992 


Posterior Mean 


0.128 


0.445 
0.109 


0.153 


Posterior SD 


0.029 


0.044 
0.031 


0.045 


t 


4.414 


10.114 
3.516 


3.400 


11. Prom pos on-campus image (n = 727) 

TERM Posterior effect probability Posterior Mean PosteriorSD_ t 
Chair22 

Chair23 

Chair24 0.994 0.206 0.049 4.204 
Chair25 

Chair26 

Chair27 

Chair28 

Chair29 

Chair30 

Chair31 1.000 0.166 Q.03). 5355 
Chair32 

Chair33 

Chair34 

Chair35 

Chair36 

Chair37 1.000 0.249 0.032 7.781 
Chair38 1.000 0.213 0.048 4.438 
Chair39 

Chair40 

Chair41 

Chair42 

Chair43 

Chair44 

Chair45 

Chair46 

Chair47 

Chair48 1.000 0.155 0.036 4.306 
Chair49 

Chair50 

Chair51 

Chair52 

Chair53 
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12. Fosters faculty talents (n = 723) 

TERM Posterior effect probability Posterior Mean Posterior SD t 
Chair22 

Chair23 

Chair24 

Chair25 

Chair26 

Chair27 

Chair28 

Chair29 

Chair30 

Chair31 

Chair32 

Chair33 0.987 0.102 0.031 3.290 
Chair34 

Chair35 1.000 0.327 0.030 10.900 
Chair36 

Chair37 

Chair38 

Chair39 

Chair40 

Chair41 0.984 0.139 0.042 3.310 
Chair42 

Chair43 

Chair44 

Chair45 

Chair46 

Chair47 

Chair48 

Chair49 

Chair50 1.000 0.133 0.029 4.586 
Chair51 

Chair52 

Chair53 
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13. Acquaints new faculty/staff (n = 718) 

TERM Posterior effect probability Posterior Mean Posterior SD t 
Chair22 

Chair23 

Chair24 

Chair25 

Chair26 

Chair27 

Chair28 0.999 0.150 0.034 4.412 
Chair29 

Chair30 

Chair31 

Chair32 

Chair33 

Chair34 

Chair35 

Chair36 

Chair37 

Chair38 

Chair39 

Chair40 

Chair41 

Chair42 0,999 0.217 0.049 4.429 
Chair43 

Chair44 

Chair45 

Chair46 

Chair47 1.000 0.238 0.052. 4.577 
Chair48 

Chair49 

Chair50 

Chair51 

Chair52 

Chair53 


14. Comm. admin. expectations (n = 730) 

TERM Posterior effect probability Posterior Mean PosteriorSD  t 
Chair22 

Chair23 

Chair24 

Chair25 

Chair26 

Chair27 

Chair28 1.000 0.123 0.027 4.556 
Chair29 

Chair30 

Chair31 093 0.102 0.030 3.400 
Chair32 0.992 0.153 0.045 3.400 
Chair33 0.995 0.131 0.038 3.447 
Chair34 

Chair35 

Chair36 

Chair37 

Chair38 

Chair39 0.957 0.115 0.043 2.674 
Chair40 

Chair41 

Chair42 

Chair43 

Chair44 

Chair45 0.955 0.148 0.054 2.741 
Chair46 

Chair47 1.000 0.291 0.045 6.467 
Chair48 

Chair49 

Chair50 

Chair51 

Chair52 

Chair53 


15. Stimulates fac. enthusiasm (n = 701) 
TERM Posterior effect probability 


Chair22 
Chair23 
Chair24 
Chair25 
Chair26 
Chair27 
Chair28 
Chair29 
Chair30 
Chair31 
Chair32 
Chair33 
Chair34 
Chair35 
Chair36 
Chair37 
Chair38 
Chair39 
Chair40 
Chair41 
Chair42 
Chair43 
Chair44 
Chair45 
Chair46 
Chair47 
Chair48 
Chair49 
Chair50 
Chair51 
Chair52 
Chair53 


1.000 


1.000 


1.000 
1.000 


Posterior Mean 


0.154 


0:137 


0.457 
0.112 


Posterior SD 


0.025 


0.028 


0.041 
0.028 


t 


6.160 


4.893 


11.146 
4.000 
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16. Guides curriculum development (n = 699) 

TERM Posterior effect probability Posterior Mean Posterior SD  t 
Chair22 1.000 0.170 0.041 4.146 
Chair23 

Chair24 

Chair25 

Chair26 

Chair27 

Chair28 

Chair29 

Chair30 

Chair31 

Chair32 

Chair33 

Chair34 

Chair35 

Chair36 1.000 0.207 0.051 4.059 
Chair37 

Chair38 

Chair39 1.000 0.253 0.037 6.838 
Chair40 

Chair41 

Chair42 

Chair43 

Chair44 1.000 0.247 0.045 5.489 
Chair45 

Chair46 

Chair47 

Chair48 

Chair49 

Chair50 

Chair51 

Chair52 

Chair53 
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17. Establishes trust (n = 745) 

TERM Posterior effect probability Posterior Mean Posterior SD t 
Chair22 

Chair23 1.000 0.110 0.024 4.583 
Chair24 

Chair25 1.000 0.374 0.031 12.065 
Chair26 

Chair27 

Chair28 

Chair29 

Chair30 

Chair31 

Chair32 0.979 0.081 0.026 3.115 
Chair33 

Chair34 1.000 0.134 0.021 6.381 
Chair35 

Chair36 0.980 0.080 0.027- - 2.963 
Chair37 

Chair38 1.000 0.124 0.026 4.769 
Chair39 

Chair40 

Chair41 

Chair42 

Chair43 

Chair44 

Chair45 

Chair46 

Chair47 

Chair48 

Chair49 

Chair50 

Chair51 

Chair52 

Chair53 


18. Prom pos off-campus image (n = 702) 

TERM Posterior effect probability Posterior Mean PosteriorSD_ t 
Chair22 

Chair23 

Chair24 

Chair25 

Chair26 

Chair27 

Chair28 

Chair29 

Chair30 1.000 0.213 0.037" °5,757 
Chair31 1.000 0.169 0.036 4.694 
Chair32 

Chair33 

Chair34 

Chair35 

Chair36 

Chair37 1.000 0.224 0.038 5.895 
Chair38 

Chair39 

Chair40 

Chair41 

Chair42 

Chair43 1.000 0.227 0,039: 5,821 
Chair44 

Chair45 

Chair46 

Chair47 

Chair48 

Chair49 

Chair50 

Chair51 

Chair52 

Chair53 


19. Rewards fac. appropriately (n = 696) 

TERM Posterior effect probability Posterior Mean PosteriorSD_ t 
Chair22 

Chair23 

Chair24 

Chair25 

Chair26 

Chair27 1.000 0.287 0.058 4.948 
Chair28 

Chair29 

Chair30 

Chair31 

Chair32 

Chair33 0.998 0.167 0.045 3.711 
Chair34 

Chair35 0.971 0.150 0.050 3.000 
Chair36 

Chair37 

Chair38 

Chair39 

Chair40 0.993 0.147 0.041 3.585 
Chair41 

Chair42 

Chair43 

Chair44 

Chair45 

Chair46 1.000 0.166 0.038 4.368 
Chair47 

Chair48 

Chair49 

Chair50 

Chair51 

Chair52 

Chair53 
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20. Assessment of student learning (n = 691) 

TERM Posterior effect probability Posterior Mean PosteriorSD t 
Chair22 

Chair23 

Chair24 

Chair25 

Chair26 

Chair27 

Chair28 0.953 0.106 0.040 2.650 
Chair29 

Chair30 

Chair31 1.000 0.145 0.035 4.143 
Chair32 

Chair33 

Chair34 

Chair35 

Chair36 

Chair37 

Chair38 

Chair39 

Chair40 1.000 0.208 0.045 4.622 
Chair41 

Chair42 

Chair43 

Chair44 1.000 0.264 0.052 5.077 
Chair45 

Chair46 

Chair47 

Chair48 

Chair49 

Chair50 

Chair51 

Chair52 

Chair53 


21. Supporting stu. recruit. and retent. (n = 692) 
TERM Posterior effect probability 


Chair22 
Chair23 
Chair24 
Chair25 
Chair26 
Chair27 
Chair28 
Chair29 
Chair30 
Chair31 
Chair32 
Chair33 
Chair34 
Chair35 
Chair36 
Chair37 
Chair38 
Chair39 
Chair40 
Chair41 
Chair42 
Chair43 
Chair44 
Chair45 
Chair46 
Chair47 
Chair48 
Chair49 
Chair50 
Chair51 
Chair52 
Chair53 


1.000 


Posterior Mean 


0.960 


Posterior SD t 


0.008 120.000 


54. Confidence in leadership (n = 758) 
TERM Posterior effect probability 


Chair22 
Chair23 
Chair24 
Chair25 
Chair26 
Chair27 
Chair28 
Chair29 
Chair30 
Chair31 
Chair32 
Chair33 
Chair34 
Chair35 
Chair36 
Chair37 
Chair38 
Chair39 
Chair40 
Chair41 
Chair42 
Chair43 
Chair44 
Chair45 
Chair46 
Chair47 
Chair48 
Chair49 
Chair50 
Chair51 
Chair52 
Chair53 


1.000 


1.000 


1.000 
1.000 
0.992 


0.963 


Posterior Mean 
0.171 


0.185 


0.239 
0.100 
0.122 


0.100 


Posterior SD 
0.033 


0.029 


0.031 
0.023 
0.035 


0.034 
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6.3/9 


7.710 
4.348 
3.486 


2.941 


55. Excellent leadership (n = 758) 

TERM Posterior effect probability Posterior Mean Posterior SD t 
Chair22 1.000 0.111 0.030 3.700 
Chair23 

Chair24 1.000 0.148 0.033 4.485 
Chair25 1.000 0.140 0.025 5.600 
Chair26 

Chair27 

Chair28 

Chair29 

Chair30 

Chair31 

Chair32 

Chair33 

Chair34 

Chair35 

Chair36 1.000 0.162 0.029 5.586 
Chair37 0.995 0.074 0.020 3.700 
Chair38 1.000 0.150 0.027 5.556 
Chair39 

Chair40 

Chair41 

Chair42 

Chair43 

Chair44 

Chair45 

Chair46 

Chair47 

Chair48 

Chair49 

Chair50 

Chair51 

Chair52 

Chair53 


APPENDIX C 


Faculty Perceptions of Chair Inter-item Correlations 
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